Arrow colors mark the three axes. Node color shows direction from the
origin. Words with the same hue sit in the same region, and gray means close to the center.
Words: 0Sentences: 0Window: w=5
appleCount: 4
3D projected coordinate [X, Y, Z]:[0.00, 0.00, 0.00]
Identical vector
These words occur in exactly the same contexts, so the model cannot tell them apart,
and they share one point in the space.
vs
cos θ (full PPMI)
angle θ
cos θ (3D only)
3D distance
Measured over all N raw dimensions vs. the 3 latent ones. When the
latent angle is much smaller, SVD has found shared context patterns that the raw counts miss.
Nearest in the latent space(after SVD, second-order):
Nearest in the raw PPMI space(before SVD, first-order):
Top co-occurring context words (raw counts):
Tap a neighbor, or shift-click another word, to measure the angle and distance between the two.
The matrix behind the picture
Every 3D point starts life as one row of this table
Rows and columns are both the vocabulary, ordered as they first appear in the corpus. A bright cell means the two
words share context far more often than chance would predict. The row for a word is its raw embedding,
a vector with one number per vocabulary item. SVD compresses those rows down to the three coordinates you see in the 3D view.
How to use this demo
Drag to rotate. Scroll or pinch to zoom.
Click or tap a word to see its nearest neighbors, before and after SVD.
Tap one of those neighbors, or shift-click any other word, to measure the angle and distance between the two.
The table button at the top right shows the raw counts and the full PPMI table.
Configure Vector Space lets you edit the sentences, change the window, try analogies, or name the axes yourself with concept seed vectors.
Drag to rotate •
Scroll to zoom •
Click a word •
Shift+click to compare