Are learning phonotactics and learning segments codependent?

Robert, Matt, Jessica and I have been having a discussion on this topic over email. Look out! Comments are interspersed.

Although I think the hypothesis y'all propose in the abstract is on the right track, I wasn't sure whether the following two facts were problematic:

1. phonotactic learning studies in babies often seem to mimic complementary distribution (e.g. Amanda's coronal stops before high vowels and labial beforemid); yet learning relies on babies still being able to discriminate coronal and labial before both types of vowel in order to treat legal and illegal trials differently. So complementary distribution (at least with this short exposure) cannot influence babies' ability to discriminate between sounds. [ac]


mm, two thoughts on this, one slightly out on a limb and the other well out on a limb. 
first thought: in my opinion, there is not a good theory of the relationship between category
formation and discrimination. most models are designed for one or the other task, and attempts to
model both generally involve taking a model of categorization and adding on an ad hoc mechanism for 
predicting discrimination. there is no apparent reason why discrimination has to decline when
categories form, and in some cases it appears not to. my co-authors may disagree with me on this 
point, but i feel like our model does not specifically address discrimination. 
second thought: i have not seen any compelling evidence that 9-month-olds "know" that there is such 
a thing as "coronal" that encompasses multiple phones. on the other hand, there is abundant
evidence that they can discriminate pretty much anything that is meaningfuly different. that's not
really in opposition to your point, i guess. more a caveat that we should be careful what
generalizations we assume the baby is assigning to her stimuli. [RD]

Ditto to what Robert said -- it's not clear that infants haven't learned
about native phonetic categories prior to the loss of sensitivity to foreign
contrasts. And b) It sounds like what you're saying is that infants can 
learn phonotactic dependencies at 4 months -- not that they know native
 language phonotactics at 4 months. So I'm not sure how that bears on the 
issue of what they have learned about the native language. Certainly being
 able to learn that sort of pattern indicates that they're likely doing that
 sort of processing in the real world. But real language almost certainly 
takes longer to learn than our miniature artificial ones. For example, 6
 month olds (and that's the youngest we've tested) show consonant learning in
my phonetic learning studies, although there's no evidence that they know
 native language consonant categories yet. [JM]

It seems to me that this sort of data is also problematic for the
segments-then-phonotactics view that we're criticizing in our work. Under a
strong version of that hypothesis, I think we'd expect that infants would be
incapable of learning anything about phonotactics until they were totally
done learning about segments (either in their native language, or in the
context of the experiment). I think these data are good news for us,
actually.
 [MG]


2. Phonotactic learning occurs long before babies start converging on their languages inventory. For instance, we've had babies learn a V-C dependency at 4 and 6 months, and a #C dependency at 7 months. [Incidentally, we found that constraints on allophones are hard to learn at later ages; so the VC dependency was on nasal vowels, and 4 and 6-mo could learn it, 11mo babies couldn't if they were learning English, but were ok with it if learning French.] [ac]

i don't think there's very clear evidence that infants haven't converged on their language's
inventory by 4-6 months. the evidence we have is that by 6 months, their within-category
discrimination DECREASES relative to across-category discrimination. which is certain proof that
there is a category there.
however, this does not mean there is no category there before: absence of evidence is not
unequivocal evidence of absence. it is entirely possible that the category onset occurs before the
decline in discrimination. this goes back to the point i was mentioning before -- until we have a
better theory of why discrimination declines with categorization, we have to be careful about how
we interpret the presence/absence of categories.
there is one really interesting study by nobuo masataka that speaks to this point -- he found that
japanese learning infants' vowel productions were significantly influenced by the most recent vowel 
their mother had produced. in particular, if mother produced /a/, the next vowel baby produces is,
on average, spectrally shifted toward /a/, and the same holds true for the other point vowels. if i 
recall correctly, this was 4-month-olds! seen from a certain theoretical lens, this could be
evidence of extremely early category formation. alternatively, it could be viewed as a means by
which categories are learned. in either case, though, it is clear evidence that infants have
already discovered aspects of the forward mapping from articulation to acoustics. [RD]

I haven't read that Masataka study but from your description I'm not sure
why you would take it to be evidence for phonetic categories. A simpler
explanation would be the latter one you mentioned -- the babies are starting
to figure out how articulation relates to acoustics.
 [JM]

probabilities

Hi all,
I have been rereading Matt's recent article on phonotactic learning with phonotactic probabilities and thinking about the upcoming workshop on cues in phonological learning and I am just wondering about the different weightings of cues in infants than adults and the different kinds of cues that we will discuss at the workshop: distributional, articulatory, acoustic, etc... In Matt's paper he found that adults were pretty insensitive to the probability variation of /s/ in onset position and I am wondering whether we might not expect different things from infants. Let me see what you all think of my predictions. While infants are clearly *very* sensitive to probabilities in the input they also might be less sensitive to articulatory ease since they don't really articulate yet (this would not be true for toddlers of course), so they might show a different performance pattern here than adults. This also begs the the question of why adults show a lack of sensitivity (perhaps something that Matt can help me with). Is it because they are less sensitive to probabilities over time or because they show a heightened sensitivity to ease of articulation?
-Amanda

Some questions on the ideas put forward by y'all

Hello everyone,

I was re-reading everyone's abstracts and thought that there were some interesting coincidences and intriguing mismatches.

For example, I might be reading Susan's wrong, but it seems that she predicts that, with a 1-to-1 mapping from articulation to acoustics, the magnitude of the change in the articulation predicts basically two levels of structure: one corresponding to slower and larger movements and one to more abrupt changes. Only the latter would, according to my interpretation of her abstract, provide acoustic cues. This matches Ying and Jeff's finding that an automatic feature extractor operating on the raw acoustic signal is good at finding manner features, but not so good with place features. On the contrary, motor cues are most useful for place of articulation, but not so great with manner.

Susan also suggests that it is not only the level of fine acoustic cues that affects speech perception, but also the other level. May this be related to Jessica, Robert and Matt (JRM)'s learner of phonotactic constraints and segments? If I get their point right, both infants and adults would rely on acoustic cues in context to discover/perceive phonological categories, and this process may go from a holistic chunk (perhaps syllable-based?) to phone-sized elements. In relation to this, I was thinking that while one may expect that articulatory experience would aid the formation of prosodic categories (like syllabic/word templates), it was at first surprising for me that it is also important for learning of segments. Now I see that this fits really well with JRM's proposal that we may learn phonotactics and segments at the same time, don't you think?

Further, this learning strategy (of phonotactics+segments) must be flexible and still active in adulthood, given that Lisa D's adults are able to learn a minimal pair of words that relies on a non-native phonotactic contrast, but her results also suggest that the presence of a minimal pair (which, through semantic cues, forces listeners to focus on the relevant acoustic cues) is a necessary condition for learning in adults. On the other hand, the presence of minimal pairs cannot be a necessary condition in toddlers (cf. Pater et al. 2004), as Ying and Jeff point out, while already in childhood semantic information is actually helpful, according to Lisa G's results. Is there a developmental trend here? May it be geared by a difference in processing abilities? (If so, would we predict that adults in a high-processing load task would behave like Pater's toddlers and have a *harder* time with the minimal pair than with the non-minimal pair?)

Another question that follows from Susan's hypothesis that these two levels interact in perception is how language experience affects this interaction. Grant's paper seems to suggest that this interaction is mediated by specific experience, such that sheer experience with one of the categories (AmEng speakers must have experience with retroflex sounds, and even with the retroflex fricative in words like 'shrill') is not enough, but speakers must have been exposed to the particular contrast that relies on the acoustic/higher level features under study. This is particularly important for early category formation: Chandan's corpus analysis looks at the correlation of pitch and VOT, which could be interpreted as respective examples of the larger and finer categories proposed by Susan. If IDS takes advantage of the apparent predilection of infants for pitch, will it have its voiced segments be primarily marked with pitch? However, given young infants' very limited articulatory abilities, we'd expect them to rely primarily on acoustic-feature based contrasts, which -now following Ying and Jeff - would predict a primacy of VOT.

A Probabilistic Clustering Approach to Feature Induction

By Ying Lin and Jeff Mielke

Phonological theories often assume that features are innate and universal, and that they characterize possible natural classes. Although this assumption is certainly useful for phonological analysis, it leaves several unresolved issues in other fields: first, from the perspective of a learner, acquiring feature-based representations of words is assumed to be trivial, and establishing distinctive features requires the availability of minimal pairs. This perspective is largely inconsistent with
the facts of language development (Charles-Luce and Luce, 1990). Second, phoneticians have long questioned the psychological reality of features, partly because no proposal has emerged about how hearers/learners can recover symbolic phonological knowledge from processes of speech perception and production (Ladefoged, 2001). These problems may be remedied by an alternative view of phonological features as being the result of learning from phonetic data.

Inspired by the behavioral study of (Maye and Gerken, 2000) on the learning of sound categories, the present study explores an inductive approach to feature discovery without resorting to minimal pairs. Our basic assumption is that learners have access to a wide variety of phonetic information, and that feature induction can proceed through recursive clustering of the appropriate data. In particular , natural classes are discovered in a coarse- to-fine manner, and features are discovered as a by-product of a hierarchy of natural classes, similar to some representational schemesin phonology (Clements, 2001; Dresher, 2003). Two kinds of data were used in our clustering experiment: the acoustic data consist of a set of segments from the TIMIT acoustic phonetic database (Garofolo, 1988). Phonetic labels of those segments have been removed prior to the experiment and are only used later in evaluation. The articulatory data are obtained with ultrasound imaging.

A set of monosyllabic words are used in the ultrasound data collection, containing a set of English consonants occurring in word-initial positions. One image was taken from the maximal constriction during the initial consonant of each word, and vocal tract is characterized by a vector of cross distances from the tongue body to an estimated position of the palate, as obtained from a palate reconstruction technique.

The clustering in the current study is based on a kind of probabilistic model called mixture model (McLachlan and Basford, 1988), and recursive clustering is realized by embedding a hierarchical structure within each mixture model. Two probabilistic, generative models are employed as the respective components of the mixture models for acoustic and articulatory data. Since acoustic speech data vary in duration and spectral properties, a mixture of Hidden Markov Models (HMM) is used to cluster the acoustic speech. For the articulatory data, since the dimensions of the vocal tract are highly correlated, a mixture of Probabilistic Principle Component Analyses (PPCA) is used to cluster the cross distances in the vocal tract. This is based on the assumption that speech articulation can be seen as perturbations along vocal tract configurations that serve as orthogonal modes, and thus can be pro jected into a lower dimensional space using the composite measures as new coordinates (Story and Titze, 1998). Our experiments using acoustic and articulatory data suggest that these two sources of information lead to the discovery of manner and place features, respectively, thus bringing support to the view of feature learning as a process of mapping from phonetics to phonology.

Phonological context as a cue to phonetic identity

By Jessica Maye, Robert Daland and Matthew Goldrick

In this talk we argue that one important cue that listeners use to identify a phonetic segment in speech is the phonological context that the segment occurs in. This is evident both in adult listeners and in infants who are still in the process of acquiring a native phonology. These facts suggest a phonological parsing system that relies on phonological as well as acoustic cues in establishing a phonological representation, both in development and throughout life.

A growing body of literature has demonstrated that phonological context affects adult listeners identification and discrimination of speech sounds (e.g., Massaro & Cohen, 1983; Pitt, 1998; Dupoux et al., 1999). The developmental literature tells a similar story. Infants begin to show the perceptual effects of the native language phonetic inventory around 9 months of age (e.g., Werker & Tees, 1984; Best, 1991; Kuhl et al., 2006). Conspicuously, around 9 months is also the age at which
they begin to show awareness of native language phonotactic constraints (Friederici & Wessels, 1993; Jusczyk et al., 1993; Mattys & Jusczyk, 2001). The developmental coincidence of these two aspects of phonological acquisition is predicted if phonetic categories and phonological regularities are two facets of a single phonological parsing system.

In this talk we will review the research demonstrating that phonological factors such as phonotactic constraints and allophony influence listeners perception of speech sounds and their ability to discriminate between acoustically similar phones. We will then present a computational model of phonological development that simultaneously learns the phones and the phonotactic constraints of a language, and will discuss how this sort of model can account for the effects of native
language phonology in phonetic identification and discrimination.

Integrality of acoustic cues depends on language- and contrast-specific experience

By Grant McGuire

Phonetic categories are generally assumed to have multiple cues available for listeners to reliably use in identification. Specifically, it has been shown that fricatives are identified through static spectral information resulting from the narrow fricative constriction and dynamic formant transition information provided by movement into or out of a preceding or following vowel (Harris 1958, Heinz and Stevens 1961, Whalen 1991) and that these cues are perceived linguistically, but not psycho-acoustically, as an integral unit (Tomiak et al. 1986). The weighting of these cues varies such that formant transitions are weighted more heavily for contrasts between spectrally similar fricatives while fricative noise is the primary cue for spectrally distinct fricatives (Harris 1958, Wagner et al. 2006). This difference in weighting changes over time as young children listen preferentially to dynamic cues (Nittrouer et al. 1993, Nittrouer and Miller 1997) leading to the general hypothesis that young children preferentially listen to large-scale dynamic changes in the speech signal (Developmental Weighting Shift, Nittrouer and Miller 1997).

This paper presents results from two experiments exploring the relative weighting and integration of sibilant fricative cues by non-native speakers. Specifically, Mandarin and American English listeners discriminated and labeled examples of Polish alveopalatal [ɕ] and retroflex [ʂ] sibilants. These two language groups were chosen as this contrast is extremely similar to a native Mandarin one (Stevens et al. 2004) but is difficult for English listeners to discriminate as they generally assimilate the two sounds to their native palato-alveolar category (Lisker 2001). The first experiment used a same~different paradigm to assess both language groups' discrimination of (A) modified naturally produced pairs of syllables having fully correlated fricative noise and formant transition cues, (B) pairs differing in a single dimension (fricative noise, formant transition), or (C) fully conflicting cue pairs. In the second experiment the same listeners labeled a two – dimensional continuum of the same modified naturally produced Polish sounds varying in fricative noise and formant transition cues where syllables had differing degrees of correlated and conflicting cues.

An analysis of subjects' reaction times demonstrates that Mandarin listeners were significantly slowed by conflicting cue stimuli in both experiments, unlike English listeners who showed no such effect. Moreover, in the labeling experiment, English listeners relied solely on formant transition information and ignored fricative noise variation. This contrasts with Mandarin listeners who use both dimensions for categorization. These results demonstrate that English listeners do not integrate the two cues and, although they are able to use fricative noise information in the discrimination experiment and they rely heavily on such cues for their native sibilant contrast, they do not use static spectral information for this contrast. This suggests that listeners do not generalize knowledge of cues beyond the contexts in which they experience them natively and that cue integration is a learned phenomenon heavily dependent on local context. The English listeners' reliance on formant transition for a novel contrast also supports Nittrouer's Developmental Weighting Shift hypothesis as a general perceptual strategy not limited to language acquisition.

Incentive to focus: Word learning helps listeners distinguish native and non-native sequences

By Lisa Davidson

Previous research in cross-language perception has shown that non-native listeners often assimilate both single phonemes and phonotactic sequences to native language categories (e.g., Best 1995, Kuhl and Iverson 1995, Dupoux et al. 1999, Flege et al. 2003). These findings suggest that it would be difficult for second language learners to overcome these phonetic barriers to learning new sounds or sequences. To study whether higher-level cues can assist learners, two experiments examined whether associating meaning with unfamiliar words assists listeners in distinguishing the phonotactically possible and unattested sequences (see also Hayes- Harb 2007 for phoneme discrimination).

In Experiment 1, American English listeners were trained on word-picture pairings of words containing a phonological contrast between CC and CəC sequences, but which were not minimal pairs (e.g., [ftake], [fətalu]). In Experiment 2, the word-picture pairings specifically consisted of minimal pairs (e.g., [ftake], [fətake]). In the test phase, listeners saw a picture and
heard words spoken by a new speaker and had to indicate which of the words matched the picture. Results showed that participants chose the accurate CC or CəC form more often when they learned minimal pairs as opposed to phonological contrast alone. Nevertheless, there was a significant asymmetry in accuracy in Experiment 1: listeners were more accurate on CəC word- picture pairings than on CC pairings. Subsequent investigation of individual listeners revealed that the participants could be divided into a high performing and a low performing group: the high performers were much more capable of learning the contrast between native and non-native words, while the low performers remained at chance. Results for high performers are shown in Figure 1 (attached).

These findings suggest that at least for high performers, learning minimal pairs provides greater incentive to distinguish non-native sequences like CC from native counterparts like CəC. These experiments can be compared to a previous AX discrimination task using the same stimuli which did not include any training on the stimuli beforehand (Davidson to appear). In the AX task, listeners were at chance in discriminating between CC and CəC tokens. Unlike evidence from the infant literature suggesting that infants may encounter processing limitations in tasks requiring them to assign meaning to contrastive sounds (Pater, Stager and Werker 2004), adults have ample resources allowing them to use meaning cues to better learn the distinction between native and non-native sequences. Furthermore, greater accuracy on phonotactically legal CəC sequences may be due to the ability to establish a more robust phonetic representation. For words learned as CC, participants seem to accept a greater variety of productions, suggesting that the native language phonological prohibition on the CC sequences used in this study hampers a detailed phonetic encoding of these items.

Attention to cues and phonological categorization: Motor contributions

By Lisa Goffman


The acquisition of phonological units relies on perceptual and motor experiences and biases. Perceptual factors have been the emphasis of much research, with infants responding to prosodic and segmental cues in the input. Such perceptual sensitivities are thought to provide a mechanism for parsing important language units, such as words and sentences.
Motor contributions to the acquisition of production units have been less well investigated than perceptual. MacNeilage and Davis (2000) have argued that motor primitives associated with the syllable emerge in the context of canonical babbling and that the segmental content of these oscillatory open-close movements is detailed over the course of development. In adult speakers, Browman and Goldstein (1992) have developed the theory of Articulatory Phonology, in which movement primitives associated with each individual articulator (e.g., tongue body, lips, velum) are coordinated, with phonological units emerging from this coordination of gestures. Current models of language production (e.g., Levelt et al., 1999) include Articulation as an element, but do not develop how this processing level interacts with the phonological level, especially during acquisition.
The research discussed in this paper attempts to bridge this gap by measuring articulatory movement output as children and adults produce various language units. The working hypothesis is that motor capacities interact with phonological units and that these interactions change over the course of development. I incorporate methodologies from speech motor control and from psycholinguistics to assess how grammatical, lexical, and phonological processing levels are linked to articulatory output.
In this work I recorded oral articulatory movements (using the Optotrak system) while children and adults produce different segments, syllables, words, and sentences. The patterning of oral movements is assessed across these different speech production tasks, as well as the variability of such patterning. Three groups of results (i.e., segmental, prosodic, and semantic) are summarized. In the segmental domain, changes in a single phoneme influence the oral movement patterning of a single articulator (Goffman & Smith, 1999). Further, changes in a single segment exert broad coarticulatory effects that cross word and phrase boundaries. Another group of findings focuses on prosody, demonstrating that children produce late developing (in English) prosodic structures with relatively stable and small and short articulatory movements. Early developing trochees are, counter-intuitively, produced with relatively poorly controlled and variable articulatory patterns (e.g., Goffman, 1999; Goffman, Heisler, & Chakraborty, 2006). That is, stress patterns that are acquired relatively early for English-learning children are actually produced with less motor precision. It may be that children either omit syllables or produce equal stress in their attempts to produce more precise iambs. Finally, data will be presented showing that 4-year-old children produce more stable articulatory movement patterns when a novel phonetic form is provided with a visual or functional referent. All of this work together provides clues into how articulatory cues contribute to children’s developing phonological units.

Abstract: The microprosody of [voice] in infant- and adult-directed speech

The acoustic cues for [voice] perception in syllable-initial stops have been shown to be predominated by the timing of voicing relative to the release of consonant closure (VOT or voice-onset time). While languages utilize VOT as the primary articulatory signature of voicing distinctions, fundamental frequency (f0) of the post-consonantal vocalic gesture has been shown to covary with VOT. Vowels following voiceless stops generally have high, falling f0 contours and low-rising f0 contours following voiced stops. Adult listeners correspondingly report CV syllables with ambiguous VOT values as voiceless when the vowel has a high, falling f0, and voiced when f0 is low and rising. The present study investigates the microprosody of voicing in infant-directed speech (IDS), which has been shown to support phonetic category formation in development. VOT/f0 covariation in the IDS of four mothers from the Brent corpus is compared to the adult-directed speech of four women from the Buckeye corpus of adult spontaneous speech. Results are discussed in light of recent research into the role of IDS in providing robust cues for phonological learning in infancy.

Factors affecting the contribution and interaction of cues

By Alejandrina Cristia, Amanda Seidl, Amelie Bernard and Kris Onishi

Although studies on adults and children frequently take into account the differential experience these populations have, further research is needed to understand the cognitive and linguistic biases that may underlie the phonological learning abilities of each population. This paper aims at raising some questions about how perceptual and linguistic experience as well as other cognitive biases may underlie the relative importance that different types of cues have in each linguistic population.

Infants' weighting of diverse kinds of cues may be expected to diverge from adults' given their different perceptual abilities and input. First, infants' articulatory and auditory systems are not fully mature. Second, infants' input is qualitatively different, not only because they are frequently exposed to infant-directed speech which may emphasize different acoustic cues than ADS, but also because they may pay attention to aspects that adults normally ignore, such as visual cues. Third, infants' working memory and processing abilities are much more limited than adults, which further entails that they cannot utilize semantic information in phonological category learning, especially at the youngest ages. Nonetheless, infants learn phonological categories and distributions and are even able to do so purely on the basis of the distribution of acoustic cues, which suggests that, to some extent, other cues may not be a {\it necessary} condition for phonological learning.

Furthermore, even within those cues that are equally available to infants and adults, infants may exhibit different learning abilities due to their more limited language experience and other cognitive biases. In one set of studies, we address the question of language experience. Would learners' ability to learn a phonotactic constraint be affected by the linguistic status of the relevant acoustic cues? In order to assess the contribution of age and language experience, we tested English-learning 4- and 11-month-olds as well as English, French and Bengali adults with a constraint on nasalized vowels. The older infants and the English adults failed to learn the pattern, suggesting that they were ignoring the cues of vowel nasalization. French and Bengali adults, for whom the contrast was phonemic, were able to learn the pattern, but so were the 4-month-olds. Thus, language experience affects attention to acoustic cues and constrains phonological learning, but, at least for young infants, it is not a necessary condition for phonological learning.


A second set of studies further investigates cognitive biases that may underlie acoustic cue-weighting. We presented infants with a clause segmentation task in which we had manipulated acoustic cues such as duration, pitch and pause and found that 4- and 6-month-olds reacted differently to these manipulations. Specifically, 6-month-olds were able to succeed at the task when only some of the cues were present but exhibited a familiarity preference (associated with harder tasks), while 4-month-olds showed a novelty preference when no cues were manipulated (suggesting that the task was easy for them) but failed when any cue was absent. We interpret these results as an indication that in speech perception infants initially pursue a holistic strategy, attending to all cues, and only later become more local processors, similarly to the development of perception in non-linguistic domains.