vector_kit 1.2.0
vector_kit: ^1.2.0 copied to clipboard
SIMD-accelerated vector math for embeddings: dot product, cosine similarity, normalization, and top-k search over packed matrices.
1.2.0 #
- The README now answers, in its first screen, why to reach for this rather than the zero-dependency route or the package that already owns the category. Both answers carry the file and line, or the issue number, that a reader can check. A "reach for it when" list and a sentence on when to skip it follow, because a page that only argues for itself is not useful for deciding.
1.1.1 #
No library code changed in this release. lib/ is byte-identical to 1.1.0.
- The screenshot caption on the package page read "millis econds". The description was a folded YAML block, the fold landed mid-word, and folding joins its lines with a space. The caption is a single-line scalar now, and the page gains a second screenshot: the platform chart, one top-k search on the VM, dart2js and dart2wasm, each measured against the same search written by hand on that target.
- The README opens with the job and its measured cost instead of a warning blockquote aimed at 1.0.4 web users. That warning grew into its own section, "Off the Dart VM", which carries the platform chart and the upgrade note. A new "What this is not" section says every query reads every row, prices what that means, and points at an approximate index for corpora past it.
- Two ratios in
doc/web-performance.mdhad drifted in the rounding: 18.53 was printed as 18 while 44.49 was printed as 45, and one cell said 1.08 where the microseconds behind it give 1.07. The table now reads 18.5x, 44.5x and 1.07x, andtool/platform_bench_chart.dartderives every ratio it draws from the same constants, which keeps the chart and the table from disagreeing again. - README-only images no longer ship in the archive. pub.dev does not read
README images out of the archive; only the two
screenshots:files have to stay in it. The download drops from 457 KB to 149 KB. example/README.mdnow explains whatfromRowsbuilds (the packed layout, with the diagram) and whatfromBytesvalidates, instead of pointing at file names.
1.1.0 #
-
The web is 15x to 41x faster. The inner loops were written around
Float32x4, which is a real SIMD type only on the Dart VM. Off it the SDK emulates it with four boxed doubles, allocating on every lane read and every arithmetic operation, and that emulation is slower than doing no SIMD at all. The kernels are now selected by a conditional import: SIMD on the VM, plain scalar loops everywhere else. Measured on 1000x384,topKCosine:1.0.4 1.1.0 native VM 79 us 77 us (unchanged) Chrome, dart2js 4,780 us 322 us Chrome, dart2wasm 12,102 us 292 us Against the same search written by hand with no package at all (257 us native, 258 us dart2js, 272 us dart2wasm), the package goes from 18x and 45x slower on the two web backends to 1.25x and 1.08x, and stays 3.3x faster on the VM. No public API changed.
-
One behavioural difference, and it is measured rather than assumed. The VM kernels accumulate in float32 because that is what their vector registers hold; the scalar kernels accumulate in double, because narrowing the running sum would mean rounding through memory on every step. Web scores therefore differ from VM scores by about 5e-9, and are slightly closer to exact. Ranking is unaffected. The new
test/cross_platform_test.dartpins the exact top-10 rows for cosine, dot and euclidean, and CI now runs the suite on the VM, dart2js and dart2wasm, so a divergence fails the build. -
A sum too large for float32 arrives as infinity on the VM and is reported by
VectorMatrix.addas an overflow. A double accumulator would carry it finitely and let the row through, so the scalar kernels fold out-of-range totals back to infinity: the error contract, not just the number, stays the same on every platform. Covered by a test. -
CI gains a
dart test -p chrome -c dart2wasmstep alongside the existing dart2js run and wasm compile.
1.0.4 #
- Ship the hand-written baseline benchmark and its numbers. 1.0.3 was published
about twenty minutes before that work landed, so the archive on pub.dev
carried a
platform_cost_test.dartwith only two tests and adoc/web-performance.mdwhose "a plain loop is close to platform-neutral" line rested on a kernel microbenchmark: one cache-resident pair of vectors, no streaming memory traffic, no per-row divide, no top-k. That is not a search, and quoting it as one overstated how the hand-written alternative does. - The third benchmark measures the real thing over the same 1000x384 corpus:
one packed
Float32List, cached row norms, a k-sized insertion top-k, no SIMD. Minimum of six native runs and four Chrome runs, spread under 2% inside each, both platforms printing the same top hit: 257 us native, 260 us on Chrome. Which puts the trade in one line: this package is 3.3x faster than that loop on the native VM and 18x slower than it on Chrome.
No library code changed in this release. lib/ is byte-identical to 1.0.3.
1.0.3 #
- Document what this package costs on the web, and test that target in CI. The
inner loops use
Float32x4, which is real SIMD only on the Dart VM; off it the type is emulated and the emulation is slower than a plain loop. Measured on 1000x384: 79 us per query natively against 4,794 us on Chrome. The package has declaredplatform:websince 1.0.0 with nothing exercising it, so the warning is now at the top of the README anddoc/web-performance.mdcarries the numbers, the cause, and the shape of the fix. - CI now runs
dart test -p chromeanddart compile wasmon every push, so the web target cannot break unnoticed. It does not assert timings; runners vary too much for a threshold to mean anything, so the benchmark prints. - Assert that quantized cosine stays inside [-1, 1].
No library code changed in this release. lib/ is byte-identical to 1.0.2.
1.0.2 #
- Add
example/README.mdfor pub.dev's Example tab (it was empty). It walks through the core example — pairwise ops, top-k cosine search, and the binary round-trip — with its real output, and points atsemantic_search.dartfor the realistic-size version. Docs only.
1.0.1 #
- Add
benchmark/quantization_benchmark.dartand a chart of what int8 quantization actually buys, measured on the same vectors: on 5,000 rows of 768 dimensions it holds the corpus in 3.7 MB instead of 14.6 MB (3.9x smaller) and keeps 99.3% of the float top-10, while search runs 4.1x slower because the byte rows cannot take the SIMD float path. The benchmark is seeded, so the memory and recall figures reproduce exactly; the README's numbers now come from it. Docs and benchmark only; no code change.
1.0.0 #
First stable release. The public API is frozen: a breaking change will not land without a major-version bump.
- Name both exports with
showclauses, so a future symbol added toops.dartorvector_matrix.dartcannot join the API by accident. The exported set is unchanged: the five free functions (cosineSimilarity,dot,euclideanDistance,normalized,normalizeInPlace) and the two classes (VectorMatrix,QuantizedMatrix, alreadyfinalsince 0.4.0).
The freeze follows an adversarial pass that ran the code against its failure
modes rather than reading them. Every one is handled with a clear
ArgumentError or FormatException, never a NaN leaking into a score, a
hang, or silent corruption: a zero vector in cosine similarity, a NaN or
infinite component, a length mismatch, k at or below zero, an empty matrix,
a topK larger than the row count, a query in the wrong dimension, a
dimension-zero matrix, truncated / empty / bad-magic bytes in fromBytes, a
quantized search over an empty matrix or one with a zero row, and a row whose
squared norm overflows single precision. The Float32List-only surface has no
async, so there is no timer or zone for a failure to escape into.
0.4.0 #
- Mark
VectorMatrixandQuantizedMatrixasfinal, ahead of a 1.0.0 freeze. Neither was designed to be subtyped: they are concrete data structures, cheap to construct, and nothing in the package, its tests, its examples or its benchmarks extends or implements either. Sealing them keeps the rest of 1.x additive, because the planned work (theQuantizedMatrixparity gaps, an ANN index) adds members to exactly these types, and every addition would otherwise break anyone who had implemented them. Addingfinalafter 1.0.0 would require a major version; removing it later would not, so this is the direction that stays open. No behaviour change.
0.3.1 #
QuantizedMatrix.topKCosineandtopKDotnow reject a query with a NaN or infinite component instead of returning a result list whose scores are all NaN.VectorMatrixalready validated this;QuantizedMatrixonly checkedkand the query length, so the same bad query that throws on one matrix type silently poisoned the ranking on the other.
0.3.0 #
- The matrix search methods take a plain
List<double>query, not only aFloat32List. An embedding straight from a model is aList<double>, so building the index withfromRowsand then callingtopKCosineused to compile on the first line and fail on the second, which is exactly the kind of seam a caller trips on. The widening is source-compatible: aFloat32Listis aList<double>, so existing calls are unchanged. Applies totopKCosine,topKDotandtopKEuclideanon bothVectorMatrixandQuantizedMatrix. example/semantic_search.dartis the real use case: a 20,000-document index of 384-dim vectors, searched, with the result measured. A top-5 query is 1.4 ms against 15.9 ms for the hand-written cosine loop (11x), and the int8QuantizedMatrixholds the index in a quarter of the memory, with the recall cost measured against the float ranking rather than assumed.
0.2.2 #
- Shorten the screenshot description. pub.dev accepts up to 200 characters but scores only those under 160, so the previous release published cleanly and quietly gave up the documentation points it was meant to earn.
0.2.1 #
- Declare the benchmark chart in
pubspec.yamlso pub.dev renders it on the package page. The chart was already in the repository and the README, but pub.dev shows only what thescreenshots:field points at, so the page a reader lands on from search opened with text where the measurement should have been.
0.2.0 #
- Add
QuantizedMatrix, an int8 form ofVectorMatrixfor corpora that no longer fit comfortably in memory. Each row is scaled so its largest component maps to 127 and stored as one byte per dimension. Measured on 5,000 rows of 768 dimensions: 14.6 MB becomes 3.7 MB, 3.92x smaller, while a search goes from 0.63 ms to 2.50 ms a query, 3.96x the time, because the byte rows cannot use the SIMD path the float rows do. It buys memory and costs throughput, which is the trade to make only when the corpus is the problem. QuantizedMatrix.fromleaves the source matrix usable, so recall can be measured against the exact ranking on real vectors. Recall@10 was 100% on the benchmark corpus, but that is an upper bound: uniformly random vectors sit far apart in high dimensions, and real embeddings cluster, which is where eight bits start confusing neighbours.
0.1.0 #
- Initial release.
- SIMD dot product, cosine similarity, Euclidean distance, and
normalization over
Float32List, with fail-fast validation of lengths and non-finite components. VectorMatrix: packed row-major storage with precomputed row norms, top-k cosine, dot product, and Euclidean search, and aVKT1binary format for serialization.