grad school, nlp
Keynote at CONLL
- harmonics in linguistics
- what does artificial language learning look like?
- english has a harmonic pattern with VP and PP, while chinese does not
- some evidence that people believe the verb phrase order generalizing it from one phrase type to another
- how to people generalize to noisy data, do people favory harmony
- no matter what, across prior patterns, people tend to favor non-harmonic
- child learners tend to favor harmonic patterns
- hypothesis: are the frequent orders of free-order languages homomorphic to the underlying structure?
- can learners infer a homomorphic order?
- even when language is non-homomorphic, they tend to favor homomorphic orders in artificial langauges
- which attributes of an object are closely associated with it?
- learners do not simply reproduce their output, their failure to correctly reproduce shows particular heuristics
- what is the relationship between imperfect learning and biases?
- noun classes/genders
- classes are not purely idiosyncratic
- typology - semantic cues are ubiquitous
- reflects salient, often highly reliable features
- children favor phonology
- possibly because they don’t know the meanings of nouns?
- inherent bias for noun-internal cues
- showing core properties for human learners, phonological vs semantics
- we’re sensitive to features of the input
- is the information entropy of a dataset reducable to the information entropy required to generate the dataset?
- socioally dominant groups
- creole is born from many different languages
- distributionally robust optimization
- does the distribution of languages within the creole actually convey information?
- can DRO help us create better LMs for creoles?
- creole-only datasets
- fasttext, using confidence scores
- arabic/chinese based creole -> how do the confidence scores for fasttext work for latin script
- why does DRO not work?
LM’s and Telicity
- telecity - whether an event has an inherent endpoint or boundary
- typically selecting of “temporal adverbial phrases”
- the object np is important for determining telecity
- is linear information in low-dimensional subspaces captured in LM’s?
- augmenting the probe model with low-rank linear projection
- are the axis of alignment in the low-rank that map to specific neurons?
- argues that LMs rely on low-dim encodings
- inlp - changes the representation maximally?
German Plural Generalization
- predicting the plural from the gender and the noun transformation
- modeling othrographic transformations
- suffixes of majority classes are overly generalized
- interesting section on intervention, intervention caused the model to start to make spelling mistakes
- prediction of plurals for arabic?
- if we train a NN to predict plurals in arabic, does that tell us anything about how people learn plurals in arabic?
Can languages models encode color?
- LM’s can capture relational knowledge, can they capture color?
- LM’s can also capture and inferred latent relationships
- color is interesting because it’s very similar to usage of color words in cognitive science
- color terms hold linguistic significance
- topological spaces of colors are somewhat projected into the color space
- color terms represent based on simple co-occurence already demonstrate correspondance, LM’s are aligned more closely