Similarity and Embeddings

The similarity pipeline embeds RHEED data as vectors, then uses those vectors to answer two different questions: which other growths look like this one, and how this growth is evolving relative to a reference.

Method	Answers
`get_embeddings()`	What are the raw vectors for this item?
`query_rheed_embeddings()`	Which other items are most similar to this one?
`get_similarity_matches()`	What are the stored top matches for this item?
`get_similarity_trajectory()`	How has similarity to a reference changed over time?

All four accept a workflow name that defaults to rheed_stationary. See Similarity for how the workflow is computed.

Find similar growths

query_rheed_embeddings() runs a k-nearest-neighbour query over the embedding index using an item’s own vectors:

from atomscale import Client

client = Client()
data_id = "44fa63b0-74da-4d25-a362-2276c80a670a"

neighbours = client.query_rheed_embeddings(data_id, top_k=10)
print(neighbours[["data_id", "similarity"]])

The result is sorted by descending similarity, where 1.0 means identical. Alongside data_id and similarity, each row carries locus columns that pin down where in each recording the match occurred: source_index, neighbor_index, real_time_seconds, and unix_time_ms.

Coarse and fine queries

The kind parameter trades precision for query count:

# Coarse: a few representative vectors per item (default)
coarse = client.query_rheed_embeddings(data_id, kind="prototype")

# Fine: one vector per window, more queries
fine = client.query_rheed_embeddings(data_id, kind="window", window_span=60.0)

window_span must match a span the data was actually embedded at. The backend caps top_k at 30. An empty DataFrame means this item has no embeddings for the given workflow and window span.

Fetch embedding vectors

Use get_embeddings() when you want the vectors themselves, for clustering, dimensionality reduction, or a custom distance metric:

emb = client.get_embeddings(data_id, window_span=60.0, kind="window")

print(emb.vectors.shape)   # (n_returned, dimension)
print(emb.count)           # total available before offset/limit
print(emb.truncated)       # True when more remain

The two kinds carry different metadata:

Kind	Vectors	Metadata
`window`	One time-resolved vector per window	`real_times`, `unix_times_ms`
`prototype`	A small representative set	`cluster_sizes`

Metadata arrays that do not apply to the returned kind are None. Page through large results with offset and limit:

first_page = client.get_embeddings(data_id, kind="window", limit=500)
second_page = client.get_embeddings(data_id, kind="window", offset=500, limit=500)

When no embeddings exist for the requested workflow and window span, the SDK emits a UserWarning and returns an empty result instead of raising, so loops over many data IDs keep running. Check len(result.vectors) before using the array.

Retrieve stored matches

get_similarity_matches() returns the top matches the platform has already computed, which is the same ranking shown in the web app:

matches = client.get_similarity_matches(data_id, window_span=60.0)
print(matches)   # columns: data_id, item_name, similarity

source_id accepts either a data ID or a physical sample ID. Set live_comparison=True to include the source entry’s still-streaming data in the comparison, and limit to cap the number of rows.

Fetch a similarity trajectory

get_similarity_trajectory() returns similarity against reference growths over time in a single call, without polling:

result = client.get_similarity_trajectory(
    source_id=data_id,
    workflow="rheed_stationary",
    last_n=200,
)

print(result.window_span)
print(result.timeseries_data.tail())

The timeseries_data DataFrame is indexed by ("Reference ID", "Time") with columns Similarity, Reference Name, UNIX Timestamp, Active, and Averaged Count. Restrict the comparison to specific references with reference_ids:

result = client.get_similarity_trajectory(
    source_id=data_id,
    reference_ids=["reference-uuid-1", "reference-uuid-2"],
)

Poll a trajectory during a growth

For a run in progress, poll instead of fetching once. The Client exposes wrappers for the four polling styles:

for frame in client.iter_poll_similarity_trajectory(source_id=data_id, interval=5.0):
    if not frame["Active"].any():
        print("Trajectory complete")
        break
    print(frame["Similarity"].iloc[-1])

Client method	Style
`iter_poll_similarity_trajectory()`	Synchronous loop
`aiter_poll_similarity_trajectory()`	Async iterator
`start_polling_similarity_trajectory_thread()`	Background daemon thread
`start_polling_similarity_trajectory_task()`	Asyncio task

Each forwards extra keyword arguments (distinct_by, until, max_polls, fire_immediately, jitter, on_error) to the underlying function in atomscale.similarity. See Poll Similarity Trajectory for the full polling walkthrough.

Introduction

Guides

References

Find similar growths

Coarse and fine queries

Fetch embedding vectors

Retrieve stored matches

Fetch a similarity trajectory

Poll a trajectory during a growth

Next steps

RHEED Features and Masks

Poll Similarity Trajectory

​Find similar growths

​Coarse and fine queries

​Fetch embedding vectors

​Retrieve stored matches

​Fetch a similarity trajectory

​Poll a trajectory during a growth

​Next steps

RHEED Features and Masks

Poll Similarity Trajectory

Find similar growths

Coarse and fine queries

Fetch embedding vectors

Retrieve stored matches

Fetch a similarity trajectory

Poll a trajectory during a growth

Next steps