跳到主要内容

Fury 0.6.0 Released

· 阅读需 5 分钟
Shawn Yang

The Apache Fury team is pleased to announce the 0.6.0 release. This is a major release that includes 35 PR from 12 distinct contributors. See the Install Page to learn how to get the libraries for your platform.

High light​

In this release, we introduced a scoped meta share mode for schema evolution in java and enabled it by default when CompatibleMode is set to Compatible:

  • This mode is 50% faster than previous KV compatible mode, and only 1/6 size of serialized payload than before.
  • It's 4x faster than protobuf, less than 1/2 serialized size of protobuf for complex object.

PerfomanceSize

Protobuf/JSON will write message fields meta and values in a KV layout, so when serializzing a list of message, they will have two issues:

  • Write meta multiple times even those message are the same type.
  • KV layout is dispersive, which is not friendly for compression.

The meta share mode will write field name&type meta of a struct only once for multiple objects of same type, which will save space and improve performance comparedto protobuf.

With meta share, we can write field name&type meta of a struct only once for multiple objects of same type, which will save space and improve performance comparedto protobuf. And we can also encode the meta into binary in advance, and use one memory copy to write it which will be much faster.

Serialize data​

  public static class NumericStruct {
    public int f1;
    public int f2;
    public int f3;
    public int f4;
    public int f5;
    public int f6;
    public int f7;
    public int f8;

    public static NumericStruct build() {
      NumericStruct struct = new NumericStruct();
      struct.f1 = 1;
      struct.f2 = 2;
      struct.f3 = 3;
      struct.f4 = 4;
      struct.f5 = 5;
      struct.f6 = 6;
      struct.f7 = 7;
      struct.f8 = 8;
      return struct;
    }
  }

  public static class NumericStructList {
    public List<NumericStruct> list;

    public static NumericStructList build() {
      NumericStructList structList = new NumericStructList();
      structList.list = new ArrayList<>(1000);
      for (int i = 0; i < 1000; i++) {
        structList.list.add(NumericStruct.build());
      }
      return structList;
    }

Result​

Performance:

Benchmark                       Mode  Cnt      Score      Error  Units
fury_deserialize                thrpt   30  49667.900 ± 3004.061  ops/s
fury_kv_compatible_deserialize  thrpt   30  33014.595 ± 3716.199  ops/s
fury_kv_compatible_serialize    thrpt   30  23915.260 ± 3968.119  ops/s
fury_serialize                  thrpt   30  63146.826 ± 2930.505  ops/s
protobuf_deserialize            thrpt   30  14156.610 ±  685.272  ops/s
protobuf_serialize              thrpt   30  10060.293 ±  706.064  ops/s

Size:

LibSerialized Payload Size
fury8077
furystrict8009
furykv48028
protobuf18000

Feature​

Bug Fix​

Others​

New Contributors​

Full Changelog: https://github.com/apache/fury/compare/v0.5.1...v0.6.0