Semantic Search Explained: How SigLIP2 Finds 'Sunset at the Beach'

Keyword search matches strings. Semantic search matches meaning. The second one used to require a server. Here is how Pixel Gallery does it on the phone, and what that costs.

Vectors, in one picture

Imagine every photo pinned on a huge wall, arranged so that similar photos sit near each other: beaches in one corner, birthday cakes in another, dogs on beaches somewhere between. A model called SigLIP2 does exactly that, except the wall has 768 dimensions instead of two. Each photo becomes a list of 768 numbers, its position on the wall. That list is an embedding.

The trick is that the same model places text on the same wall. The phrase “dog at the beach” lands near the photos of dogs at beaches. Search is then geometry: find the photos closest to the query’s position.

What happens during indexing

The app opens each photo, resizes it to the model’s input size, and runs SigLIP2 through ONNX Runtime on the CPU. For videos, it samples a handful of frames and embeds each one. The resulting vectors go into the encrypted metadata database. On a recent flagship this takes about two seconds per image, so a library of ten thousand photos takes a few charging sessions to index fully. Results are usable as soon as part of the library is done.

What happens when you search

Your query is embedded the same way, once. The app computes similarity between the query vector and every stored vector, which is fast enough to feel instant. The top matches are the semantic results.

They are not the only results. Pixel Gallery also runs its classic full-text search over captions, labels, recognized text and file names, and fuses the two rankings. A query with an exact word, such as a city name, gets both the semantic neighbors and the exact hits.

What it cannot do

The model knows what things look like, not who anyone is. “Grandpa on the porch” fails until you name Grandpa in People; after that, the name is an exact-match term and the query works. The model also has a fixed vocabulary of visual concepts; very specific or technical queries may fall back to keyword matches.

Costs

• About 375 MB for the model, downloaded once from Google Play as an asset pack.
• Roughly two seconds per photo to index, in the background.
• Enough RAM to hold the model, so the feature is hidden on low-memory devices.

Why it is free

Semantic search is part of what makes the gallery smart, and everything that makes it smart is free. Premium is reserved for power features that build on top. To set it up, follow Enable semantic search.