Note that there is no meeting of the Semantics Research Group this week.
Tuesday, March 19, 1:30 PM - 3:30 PM, PT266
Computational Linguistics Group, Department of Computer Science
Kawin Ethayarajh (Ph.D., Department of Computer Science): "Towards understanding linear word analogies."
A surprising property of word vectors is that vector algebra can often be used to solve word analogies (e.g., king - man + woman = queen). However, it is unclear why - and when - linear operators correspond to non-linear embedding models such as skip-gram with negative sampling (SGNS). We provide a rigorous explanation of this phenomenon without making the strong assumptions that past theories have made about the vector space and word distribution. Our theory has several implications. Past work has conjectured that linear structures exist in vector spaces because relations can be represented as ratios; we prove that this holds for SGNS. We provide novel justification for the addition of SGNS word vectors by showing that it automatically down-weights the more frequent word, as weighting schemes do ad hoc. Lastly, we offer an information theoretic interpretation of Euclidean distance in vector spaces, justifying its use in capturing word dissimilarity.
Friday, March 22, 10:00 AM - 11:30 AM, SS560A
Language Variation and Change Research Group
Guest speaker: Heike Pichler (Newcastle University): "'This is what happened, right?': Sex differences in narrative tagging."
Friday, March 22, 11:30 AM - 1:00 PM, SS560A
Phonology Research Group
TBA
No comments:
Post a Comment