跳到主要内容

21 篇博文 含有标签「fury」

查看所有标签

· 阅读需 6 分钟
Shawn Yang

We're excited to announce the release of Fury v0.5.0. This release incorporates a myriad of improvements, bug fixes, and new features across multiple languages including Java, Golang, Python and JavaScript. It further refines Fury's performance, compatibility, and developer experience.

New Features​

Specification​

  • Introduced fury cross-language serialization specification (#1413, #1508)
  • Introduced xlang type mapping (#1468)
  • Introduced fury java specification (#1240)
  • Introduced meta string encoding specification (#1565, #1513, #1517)

Java​

  • Support for compatible mode with GraalVM (#1586, #1587).
  • Support unexisted array/enum classes and enabled deserializeUnexistedClass by default (#1569, #1575).
  • meta string encoding algorithm in java (#1514, #1568, #1516, #1565)
  • Support meta string encoding for classname and package name (#1527)
  • native streaming mode deserialization (#1451, #1551)
  • native channel stream reader (#1483)
  • Support registration in thread safe fury (#1280)
  • Implement fury logger and remove slf4j library (#1485, #1494, #1506, #1492)
  • Support adjust logger level dynamically (#1557)
  • Support jdk proxy serialization for graalvm (#1379)
  • Specify JPMS module names (#1343)
  • Align string array to collection protocol v2 (#1228)

JavaScript​

  • Align implementation to new Xlang protocol (#1487)
  • Implement Xlang map (#1549)
  • Implemented xlang map code generator (#1571)
  • Added magic number feature for better serialization control (#1550).
  • Support oneof (#1348)
  • create zero-copy buffer when convert (#1386)
  • Implement the collection protocol (#1337)
  • Implement Enum (#1321)
  • compress numbers (#1290)

C++​

  • Support optional fields/elements in RowEncoder (#1223)
  • Support mapping types for RowEncodeTrait (#1247)

Golang​

  • Implemented Fury meta string encoding algorithm (#1566).
  • concat meta string len with flags (#1517)

Enhancements​

Java​

  • Improved buffer growth strategy to support larger data sizes for serialization (#1582).
  • Performance optimizations for MetaStringDecoder and various serialization processes (#1568, #1511, #1493).
  • concat write classname flag with package name (#1523)
  • concat meta string len with flags (#1517)
  • fastpath for read/write small varint in range [0,127] (#1503)
  • optimize read float/double for jvm jit inline (#1472)
  • replace Guava's TypeToken with self-made (#1553)
  • Remove basic guava API usage (#1244)
  • optimize fury creation speed (#1511)
  • optimize string serialization by concat coder and length (#1486)
  • carry read objects when deserialization fail for better trouble shooting (#1420)
  • implement define_class insteadof using javaassist (#1422)
  • avoid recompilation when gc happens for memory pressure (#1411, #1585)
  • Fix immutable collection ref tracking (#1403)
  • reduce fury caller stack (#1496)
  • Extract BaseFury interface (#1382)
  • refine collection builder util (#1334)
  • disable async compilation for graalvm (#1222)
  • refine endian check code size in buffer (#1501)
  • generate list fori loop instead of iterator loop for list serialization (#1493)
  • Reduce unsafeWritePositiveVarLong bytecode size. (#1491)
  • Reduce unsafePutPositiveVarInt bytecode size. (#1490, #1489)
  • optimize read char/short jvm jit inline (#1471)
  • reduce code size of read long to optimize jvm jit inline (#1470)
  • reduce readInt/readVarInt code size for for jvm jit inline (#1469)
  • refactor readVarUint32 algorithm (#1462)
  • rewrite readVarUint64 algorithm (#1463)

JavaScript​

  • Make PlatformBuffer available if has Buffer polyfill (#1373)
  • enhance performance 64bits number (#1320)
  • Refactor & Compress Long (#1313)
  • Improve tag write performance (#1241)
  • Add more methods for BinaryReader (#1231)
  • Implements tuple serializer (#1216)

Python​

  • concat meta string len with flags (#1517)

Bug Fix​

Java​

  • Fix bytebuffer no such method error (#1580)
  • Prevent exception in ObjectArray.clearObjectArray() (#1573)
  • Fix slf4j on graalvm (#1432)
  • Fix illegal classname caused by negative hash (#1436)
  • Fix BigDecimal serializer (#1431)
  • Fix BigInteger serialization (#1479)
  • Fix type conflict in method split (#1371)
  • Fix CodeGen Name conflicts when omitting java.lang prefix #1363 (#1366)
  • Fix ClassLoader npe in loadOrGenCodecClass (#1346)
  • Fix big buffer trunc (#1402)
  • Make Blacklist detection is also performed when the Class is registered. (#1398)
  • avoid big object graph cause buffer take up too much memory (#1397)
  • Fix get static field by unsafe (#1380)
  • Fix javax package for accessor codegen (#1388)
  • Fix nested collection cast for scala/java (#1333)
  • Fix References within InvocationHandler (#1365)
  • Allow partial read of serialized size from InputStream (#1391)
  • add potential missing bean class-loader (#1381)
  • Fix polymorphic array serialization (#1324)
  • Fix nested collection num elements (#1306)
  • Fix collection init size typo (#1342)
  • Clear extRegistry.getClassCtx if generate serializer class failed (#1221)

Rust​

  • Fix memory errors caused by casting (#1372)
  • Fix incorrect cast (#1345)

Miscellaneous​

  • Numerous code cleanups, refactorings, and internal improvements across all supported languages to enhance code quality and maintainability.
  • Moved various utilities into more appropriate packages to improve code organization and readability (#1584, #1583, #1578).
  • rename MemoryBuffer read/write/put/getType with read/write/put/getTypeNumber (#1480, #1464, #1505, #1500)
  • extract public Fury methods to BaseFury (#1467)
  • Optimize Class ID allocation. (#1406)
  • refine Collection util data structure (#1287) (#1288)
  • Improve Status by using unique_ptr (#1234)
  • Improve FormatTimePoint by removing sstream (#1233)
  • Drop optional chaining expression (#1338)

New Contributors​

Acknowledgements​

Thanks @chaokunyang @theweipeng @PragmaTwice @LiangliangSui @nandakumar131 @Munoon @qingoba @vesense @liuxiaocs7 @mtf90 @bowin @cn-at-osmit @Maurice-Betzel @phogh @laglangyue @tommyettinger @huisman6 @pixeeai

A big thank you to all our contributors who have worked hard on this release. Your contributions, whether through code, documentation, or issue reporting, are really appreciated.

Full Changelog: https://github.com/apache/fury/compare/v0.4.1...v0.5.0

· 阅读需 12 分钟
Shawn Yang

Apache Fury (incubating) is a multi-language serialization framework powered by JIT dynamic compilation and zero copy. It implements multi-language SDKs: Java, Python, Golang, JavaScript, Rust, C++. It provides automatic multi-language objects serialization features, and 170x speedup compared to JDK serialization.

· 阅读需 2 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm pleased to announce the 0.4.1 release of the Fury: https://github.com/alipay/fury/releases/tag/v0.4.1. With this release, Fury support rust row format now. C++ row format has been enhanced too, now iterable types can be encoded to fury row format. Please try it out and share your feedbacks with us.

Author: chaokunyang

I'm pleased to announce the 0.4.1 release of the Fury: https://github.com/alipay/fury/releases/tag/v0.4.1. With this release, Fury support rust row format now. C++ row format has been enhanced too, now iterable types can be encoded to fury row format too.

Highlight​

  • [Rust] Support row format
  • [C++] Support iterable types for RowEncoder
  • [JavaScript] Support partial record
  • [Java] Fix JIT error in corner case, now Fury can generate serializer for every class

What's Changed​

New Contributors​

Full Changelog: https://github.com/alipay/fury/compare/v0.4.0...v0.4.1

· 阅读需 3 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm pleased to announce the 0.4.0 release of the Fury. With this release, GraalVM native image and C++ row format automatic encoder based on compile-time reflection are supported. Please try it out and share your feedbacks with us.

Author: chaokunyang

I'm pleased to announce the 0.4.0 release of the Fury. With this release, GraalVM native image and C++ row format automatic encoder based on compile-time reflection are supported. Please try it out and share your feedbacks with us.

Highlight​

  • [Java] Support Graalvm native image. The implementation will generate all serialization code at image build time, the runtime will be extremely fast, see fury graalvm usage doc
  • [Java] Fury vs JDK benchmark on Graalvm native image
  • [Scala] Serialization support for package scoped object
  • [C++] Reflection support by macro/template programing
  • [C++] Automatic row format encoder

What's Changed​

Full Changelog: https://github.com/alipay/fury/compare/v0.3.1...v0.4.0

· 阅读需 3 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm pleased to announce the 0.3.1 release of the Fury. With this release, fury supports python 3.11&3.12, droped python 3.6 support. Multiple scala serialization JIT optimization are included, and java serialization compatibility is improved too.

Author: chaokunyang

I'm pleased to announce the 0.3.1 release of the Fury. With this release, fury supports python 3.11&3.12, droped python 3.6 support. Multiple scala serialization JIT optimization are included, and java serialization compatibility is improved too.

Highlight​

  • Support python 3.11 and 3.12, drop python 3.6 support
  • Refactor collection serialization framework to support writeReplace JIT
  • Integrate scala collection with fury java collection framework
  • Support scala collection jit serialization
  • Support shim dispatcher to resolve compatibility problems for common used classes
  • Use lastest arrow 14 version for row format in Java and Python

What's Changed​

New Contributors​

Full Changelog: https://github.com/alipay/fury/compare/v0.3.0...v0.3.1

· 阅读需 5 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm pleased to announce the 0.3.0 release of the Fury. With this release, fury supports all scala 2/3 objects serializaiton now, including: case/pojo/object/option/tuple/collecton/enum and other types. case/pojo/object are tightly integrated with fury JIT. Fury will generate highly-optimized serializer by generate serialize code at runtime to speed up serializaiton. The serialization for those objects will be extremely fast.

Author: chaokunyang

I'm pleased to announce the 0.3.0 release of the Fury. With this release, fury supports all scala 2/3 objects serializaiton now, including: case/pojo/object/option/tuple/collecton/enum and other types. case/pojo/object are tightly integrated with fury JIT. Fury will generate highly-optimized serializer by generate serialize code at runtime to speed up serializaiton. The serialization for those objects will be extremely fast.

For a long time, scala serialization is always tricky, only reliable way is JDK serialization. But it’s too slow, and the serialized size is too large. Although we have frameworks such as chill, but the support for scala types are limited, and the performance is limited too.Now with fury scala, you can serialize any scala objects you want and get extremly fast performance. Please try it out and let us know if you have any issues.

Scala Serialization Guide​

Install dependecy:

libraryDependencies += "org.furyio" % "fury-core" % "0.3.0"

Creating fury:

val fury = Fury.builder()
 .withScalaOptimizationEnabled(true)
 .requireClassRegistration(false)
 .withRefTracking(true)
 .build()

Serialize case objects:

case class Person(github: String, age: Int, id: Long)
val p = Person("https://github.com/chaokunyang", 18, 1)
println(fury.deserialize(fury.serialize(p)))
println(fury.deserializeJavaObject(fury.serializeJavaObject(p)))

Serialize singleton objects:

object singleton {
}
val o1 = fury.deserialize(fury.serialize(singleton))
val o2 = fury.deserialize(fury.serialize(singleton))
println(o1 == o2)

Serialize collection objects:

val seq = Seq(1,2)
val list = List("a", "b") val map = Map("a" -> 1, "b" -> 2)
println(fury.deserialize(fury.serialize(seq)))
println(fury.deserialize(fury.serialize(list)))
println(fury.deserialize(fury.serialize(map)))

Serialize enum:

enum Color { case Red, Green, Blue }
println(fury.deserialize(fury.serialize(Color.Green)))

Highlight​

  • [Scala] Support scala serialization: case/object/tuple/string/collection/enum/basic all supported
  • [Scala] Add scala user documentation
  • [Scala] add optimized scala singleton object serializer
  • [Java] Make java.io.Externalizable compatible with Java writeReplace/readResolve API
  • [Java] Integrate fury with dubbo https://github.com/apache/dubbo-spi-extensions/pull/226
  • [Java] support bytes string serialization for jdk8 with JDK17 runtime

BugFix​

  • [Java] Allow for InputStream not reading entire length
  • [Java] Use ReflectionUtils.getCtrHandle() for non-public constructor in ExternalizableSerializer
  • [Java] fix jdk compatible serialization for inheritance

What's Changed​

New Contributors​

Full Changelog: https://github.com/alipay/fury/compare/v0.2.1...v0.3.0

· 阅读需 2 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm pleased to announce the 0.2.1 release of the Fury. This version provides beter jdk17+ record support and bettern jit for private classes.

Author: chaokunyang

I'm pleased to announce the 0.2.1 release of the Fury. This version provides beter jdk17+ record support and bettern jit for private classes. Feel free to try it out and let me know if you have any issues.

Highlight​

  • [Go] Support tiny go compilation
  • [Java] Support private JDK17+ record class JIT serialization

Bug fix​

  • Fix jit error for register private serializers
  • Upgrade janino version to fix package name conflict with classname

What's Changed​

New Contributors​

Full Changelog: https://github.com/alipay/fury/compare/v0.2.0...v0.2.1

· 阅读需 8 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

I'm so excited to announce the 0.2.0 release of the Fury. This a very exciting version of fury. With this version, we have 1X speedup, 30%~50% smaller serialized size. And now we are the fastest serialization framework in the JVM-Serializers. At the same time, fury has fully support for JDK17/21, I supported JDK17+ record JIT serialization, JDK9+ ImmutableList JIT serialization. The serialization is blazing fast, please try it out. And we also open sourced Fury GO, the first golang serialization framework which supports circular reference and interface polymorphismm, feel free to try it out and let me know if you have any issues.

· 阅读需 1 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

Author: chaokunyang

I'm pleased to announce the 0.1.2 release of Fury. This release is a enhanced verson based on fury 0.1.1. It keeps binary compatibility with 0.1.0/0.1.1.

What's Changed​

  • [Java] Skip toString in annotation invocation handler readObject (#746)
  • [Python] Use cloudpickle for local function serialization (#914)
  • [Python] fix row format getter (#915)

Full Changelog: https://github.com/alipay/fury/compare/v0.1.0...v0.1.1

· 阅读需 2 分钟
Shawn Yang
信息

This release was made before Fury joined the Apache Incubator and thus it's a non-ASF release.

Author: chaokunyang

I'm pleased to announce the 0.1.1 release of Fury. This release is a enhanced verson based on fury 0.1.0. It keeps binary compatibility with 0.1.0.

Highlights​

What's Changed​

  • [Java] Dubbo fury integration: https://github.com/fury-project/dubbo-serialization-fury
  • [Java] fix isSecure check for xlang in java (#822)
  • [Java] register empty object by default (#829)
  • [Java] upgrade guava for(#831)
  • [Python] Support pyarrow6 bazel build (#859)
  • [Python] add missing shared lib for python (#862)
  • [Java] optimize buffer copy (#870)
  • [Doc] Fix row format doc (#873)
  • [Java] fix ofHashMap (#877)
  • [Java] add jdk tools.jar for tests (#881)
  • [Java] Fix lazy bean serializer (#883)
  • [Java] print exception for fury creation (#885)
  • [Java] remove getCurrentFury API (#887)
  • [Java] Add fury map iterator api (#889)
  • [Java] Add class checker API (#890)
  • [Python] Fix python linux deploy missing shared library files (#892)
  • [[Java] refine AllowListChecker (#893)

Full Changelog: https://github.com/alipay/fury/compare/v0.1.0...v0.1.1