Utterance structure - Festival Speech Synthesis System

14.1 Utterance structure

Festival's basic object for synthesis is the utterance. An represents some chunk of text that is to be rendered as speech. In general you may think of it as a sentence but in many cases it wont actually conform to the standard linguistic syntactic form of a sentence. In general the process of text to speech is to take an utterance which contains a simple string of characters and convert it step by step, filling out the utterance structure with more information until a waveform is built that says what the text contains.

The processes involved in conversion are, in general, as follows

Tokenization: Converting the string of characters into a list of tokens. Typically this means whitespace separated tokesn of the original text string.
Token identification: identification of general types for the tokens, usually this is trivial but requires some work to identify tokens of digits as years, dates, numbers etc.
Token to word: Convert each tokens to zero or more words, expanding numbers, abbreviations etc.
Part of speech: Identify the syntactic part of speech for the words.
Prosodic phrasing: Chunk utterance into prosodic phrases.
Lexical lookup: Find the pronucnation of each word from a lexicon/letter to sound rule system including phonetic and syllable structure.
Intonational accents: Assign intonation accents to approrpiate syllables.
Assign duration: Assign duration to each phone in the utterance.
Generate F0 contour (tune): Generate tune based on accents etc.
Render waveform: Render waveform from phones, duration and F) target values, this itself may take several steps including unit selection (be they diphones or other sized units), imposition of dsesired prosody (duration and F0) and waveform reconstruction.

The number of steps and what actually happens may vary and is dependent on the particular voice selected and the utterance's type, see below.

Each of these steps in Festival is achived by a module which will typically add new information to the utterance structure.

An utterance structure consists of a set of items which may be part of one or more relations. Items represent things like words and phones, though may also be used to represent less concrete objects like noun phrases, and nodes in metrical trees. An item contains a set of features, (name and value). Relations are typically simple lists of items or trees of items. For example the the Word relation is a simple list of items each of which represent a word in the utterance. Those words will also be in other relations, such as the SylStructure relation where the word will be the top of a tree structure containing its syllables and segments.

Unlike previous versions of the system items (then called stream items) are not in any particular relations (or stream). And are merely part of the relations they are within. Importantly this allows much more general relations to be made over items that was allowed in the previous system. This new architecture is the continuation of our goal of providing a general efficient structure for representing complex interrelated utterance objects.

The architecture is fully general and new items and relations may be defined at run time, such that new modules may use any relations they wish. However within our standard English (and other voices) we have used a specific set of relations ass follows.

Token: a list of trees. This is first formed as a list of tokens found in a character text string. Each root's daughters are the Word's that the token is related to.
Word: a list of words. These items will also appear as daughters (leaf nodes) of the Token relation. They may also appear in the Syntax relation (as leafs) if the parser is used. They will also be leafs of the Phrase relation.
Phrase: a list of trees. This is a list of phrase roots whose daughters are the Word's within those phrases.
Syntax: a single tree. This, if the probabilistic parser is called, is a syntactic binary branching tree over the members of the Word relation.
SylStructure: a list of trees. This links the Word, Syllable and Segment relations. Each Word is the root of a tree whose immediate daughters are its syllables and their daughters in turn as its segments.
Syllable: a list of syllables. Each member will also be in a the SylStructure relation. In that relation its parent will be the word it is in and its daughters will be the segments that are in it. Syllables are also in the Intonation relation giving links to their related intonation events.
Segment: a list of segments (phones). Each member (except silences) will be leaf nodes in the SylStructure relation. These may also be in the Target relation linking them to F0 target points.
IntEvent: a list of intonation events (accents and boundaries). These are related to syllables through the Intonation relation as leafs on that relation. Thus their parent in the Intonation relation is the syllable these events are attached to.
Intonation: a list of trees relating syllables to intonation events. Roots of the trees in Intonation are Syllables and their daughters are IntEvents.
Wave: a single item with a feature called wave whose value is the generated waveform.

This is a non-exhaustive list some modules may add other relations and not all utterance may have all these relations, but the above is the general case.