Some challenges of speech synthesis

Text normalization challenges
The process of normalizing text is rarely straightforward. Texts are full of heteronyms, numbers, and abbreviations that all require expansion into a phonetic representation. There are many spellings in English which are pronounced differently based on context. For example, “My latest project is to learn how to better project my voice” contains two pronunciations of “project”.
Most text-to-speech (TTS) systems do not generate semantic representations of their input texts, as processes for doing so are not reliable, well understood, or computationally effective. As a result, various heuristic techniques are used to guess the proper way to disambiguate homographs, like examining neighboring words and using statistics about frequency of occurrence.
Deciding how to convert numbers is another problem that TTS systems have to address. It is a simple programming challenge to convert a number into words, like “1325″ becoming “one thousand three hundred twenty-five.” However, numbers occur in many different contexts; when part of an address, “1325″ should likely be read as “thirteen twenty-five”, or, when part of a social security number, as “one three two five”. A TTS system can often infer how to expand a number based on surrounding words, numbers, and punctuation, and sometimes the system provides a way to specify the context if it is ambiguous.[citation needed]
Similarly, abbreviations can be ambiguous. For example, the abbreviation “in” for “inches” must be differentiated from the word “in”, and the address “12 St John St.” uses the same abbreviation for both “Saint” and “Street”. TTS systems with intelligent front ends can make educated guesses about ambiguous abbreviations, while others provide the same result in all cases, resulting in nonsensical (and sometimes comical) outputs.

Text-to-phoneme challenges
Speech synthesis systems use two basic approaches to determine the pronunciation of a word based on its spelling, a process which is often called text-to-phoneme or grapheme-to-phoneme conversion (phoneme is the term used by linguists to describe distinctive sounds in a language). The simplest approach to text-to-phoneme conversion is the dictionary-based approach, where a large dictionary containing all the words of a language and their correct pronunciations is stored by the program. Determining the correct pronunciation of each word is a matter of looking up each word in the dictionary and replacing the spelling with the pronunciation specified in the dictionary. The other approach is rule-based, in which pronunciation rules are applied to words to determine their pronunciations based on their spellings. This is similar to the “sounding out”, or synthetic phonics, approach to learning reading.
Each approach has advantages and drawbacks. The dictionary-based approach is quick and accurate, but completely fails if it is given a word which is not in its dictionary.[citation needed] As dictionary size grows, so too does the memory space requirements of the synthesis system. On the other hand, the rule-based approach works on any input, but the complexity of the rules grows substantially as the system takes into account irregular spellings or pronunciations. (Consider that the word “of” is very common in English, yet is the only word in which the letter “f” is pronounced [v].) As a result, nearly all speech synthesis systems use a combination of these approaches.
Some languages, like Spanish, have a very regular writing system, and the prediction of the pronunciation of words based on their spellings is quite successful.[citation needed] Speech synthesis systems for such languages often use the rule-based method extensively, resorting to dictionaries only for those few words, like foreign names and borrowings, whose pronunciations are not obvious from their spellings. On the other hand, speech synthesis systems for languages like English, which have extremely irregular spelling systems, are more likely to rely on dictionaries, and to use rule-based methods only for unusual words, or words that aren’t in their dictionaries.

Evaluation challenges
It is very difficult to evaluate speech synthesis systems consistently because there is no subjective criterion and usually different organizations use different speech data. The quality of a speech synthesis system highly depends on the quality of recording. Therefore, evaluating speech synthesis systems is almost the same as evaluating the recording skills.
Recently researchers start evaluating speech synthesis systems using the common speech dataset.[19] This may help people to compare the difference between technologies rather than recordings.

Speech Synthesis

Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware. A text-to-speech (TTS) system converts normal language text into speech; other systems render symbolic linguistic representations like phonetic transcriptions into speech.
Synthesized speech can be created by concatenating pieces of recorded speech that are stored in a database. Systems differ in the size of the stored speech units; a system that stores phones or diphones provides the largest output range, but may lack clarity. For specific usage domains, the storage of entire words or sentences allows for high-quality output. Alternatively, a synthesizer can incorporate a model of the vocal tract and other human voice characteristics to create a completely “synthetic” voice output.

The quality of a speech synthesizer is judged by its similarity to the human voice, and by its ability to be understood. An intelligible text-to-speech program allows people with visual impairments or reading disabilities to listen to written works on a home computer. Many computer operating systems have included speech synthesizers since the early 1980s.

A text-to-speech system (or “engine”) is composed of two parts: a front-end and a back-end. The front-end has two major tasks. First, it converts raw text containing symbols like numbers and abbreviations into the equivalent of written-out words. This process is often called text normalization, pre-processing, or tokenization. The front-end then assigns phonetic transcriptions to each word, and divides and marks the text into prosodic units, like phrases, clauses, and sentences. The process of assigning phonetic transcriptions to words is called text-to-phoneme or grapheme-to-phoneme conversion .[3] Phonetic transcriptions and prosody information together make up the symbolic linguistic representation that is output by the front-end. The back-end—often referred to as the synthesizer—then converts the symbolic linguistic representation into sound.

The most important qualities of a speech synthesis system are naturalness and Intelligibility. Naturalness describes how closely the output sounds like human speech, while intelligibility is the ease with which the output is understood. The ideal speech synthesizer is both natural and intelligible. Speech synthesis systems usually try to maximize both characteristics.
The two primary technologies for generating synthetic speech waveforms are concatenative synthesis and formant synthesis. Each technology has strengths and weaknesses, and the intended uses of a synthesis system will typically determine which approach is used.

隐含马尔可夫模型在语言处理中的应用

前言：隐含马尔可夫模型是一个数学模型，到目前为之，它一直被认为是实现快速精确的语音识别系统的最成功的方法。复杂的语音识别问题通过隐含马尔可夫模型能非常简单地被表述、解决，让我不由由衷地感叹数学模型之妙。

自然语言是人类交流信息的工具。很多自然语言处理问题都可以等同于通信系统中的解码问题 — 一个人根据接收到的信息，去猜测发话人要表达的意思。这其实就象通信中，我们根据接收端收到的信号去分析、理解、还原发送端传送过来的信息。以下该图就表示了一个典型的通信系统：

其中 s1，s2，s3…表示信息源发出的信号。o1, o2, o3 … 是接受器接收到的信号。通信中的解码就是根据接收到的信号 o1, o2, o3 …还原出发送的信号 s1，s2，s3…。

其实我们平时在说话时，脑子就是一个信息源。我们的喉咙（声带），空气，就是如电线和光缆般的信道。听众耳朵的就是接收端，而听到的声音就是传送过来的信号。根据声学信号来推测说话者的意思，就是语音识别。这样说来，如果接收端是一台计算机而不是人的话，那么计算机要做的就是语音的自动识别。同样，在计算机中，如果我们要根据接收到的英语信息，推测说话者的汉语意思，就是机器翻译； 如果我们要根据带有拼写错误的语句推测说话者想表达的正确意思，那就是自动纠错。

那么怎么根据接收到的信息来推测说话者想表达的意思呢？我们可以利用叫做“隐含马尔可夫模型”（Hidden Markov Model）来解决这些问题。以语音识别为例，当我们观测到语音信号 o1,o2,o3 时，我们要根据这组信号推测出发送的句子 s1,s2,s3。显然，我们应该在所有可能的句子中找最有可能性的一个。用数学语言来描述，就是在已知 o1,o2,o3,…的情况下，求使得条件概率
P (s1,s2,s3,…|o1,o2,o3….) 达到最大值的那个句子 s1,s2,s3,…

当然，上面的概率不容易直接求出，于是我们可以间接地计算它。利用贝叶斯公式并且省掉一个常数项，可以把上述公式等价变换成

P(o1,o2,o3,…|s1,s2,s3….) * P(s1,s2,s3,…)
其中
P(o1,o2,o3,…|s1,s2,s3….) 表示某句话 s1,s2,s3…被读成 o1,o2,o3,…的可能性, 而
P(s1,s2,s3,…) 表示字串 s1,s2,s3,…本身能够成为一个合乎情理的句子的可能性，所以这个公式的意义是用发送信号为 s1,s2,s3…这个数列的可能性乘以 s1,s2,s3…本身可以一个句子的可能性，得出概率。

（读者读到这里也许会问，你现在是不是把问题变得更复杂了，因为公式越写越长了。别着急，我们现在就来简化这个问题。）我们在这里做两个假设：

第一，s1,s2,s3,… 是一个马尔可夫链，也就是说，si 只由 si-1 决定 (详见系列一)；
第二， 第 i 时刻的接收信号 oi 只由发送信号 si 决定（又称为独立输出假设, 即 P(o1,o2,o3,…|s1,s2,s3….) = P(o1|s1) * P(o2|s2)*P(o3|s3)…。
那么我们就可以很容易利用算法 Viterbi 找出上面式子的最大值，进而找出要识别的句子 s1,s2,s3,…。

满足上述两个假设的模型就叫隐含马尔可夫模型。我们之所以用“隐含”这个词，是因为状态 s1,s2,s3,…是无法直接观测到的。

隐含马尔可夫模型的应用远不只在语音识别中。在上面的公式中，如果我们把 s1,s2,s3,…当成中文，把 o1,o2,o3,…当成对应的英文，那么我们就能利用这个模型解决机器翻译问题； 如果我们把 o1,o2,o3,…当成扫描文字得到的图像特征，就能利用这个模型解决印刷体和手写体的识别。

P (o1,o2,o3,…|s1,s2,s3….) 根据应用的不同而又不同的名称，在语音识别中它被称为“声学模型” (Acoustic Model)， 在机器翻译中是“翻译模型” (Translation Model) 而在拼写校正中是“纠错模型” (Correction Model)。 而P (s1,s2,s3,…) 就是我们在系列一中提到的语言模型。

在利用隐含马尔可夫模型解决语言处理问题前，先要进行模型的训练。 常用的训练方法由伯姆（Baum）在60年代提出的，并以他的名字命名。隐含马尔可夫模型在处理语言问题早期的成功应用是语音识别。七十年代，当时 IBM 的 Fred Jelinek (贾里尼克) 和卡内基·梅隆大学的 Jim and Janet Baker (贝克夫妇，李开复的师兄师姐) 分别独立地提出用隐含马尔可夫模型来识别语音，语音识别的错误率相比人工智能和模式匹配等方法降低了三倍 (从 30% 到 10%)。 八十年代李开复博士坚持采用隐含马尔可夫模型的框架， 成功地开发了世界上第一个大词汇量连续语音识别系统 Sphinx。

我最早接触到隐含马尔可夫模型是几乎二十年前的事。那时在《随机过程》（清华“著名”的一门课）里学到这个模型，但当时实在想不出它有什么实际用途。几年后，我在清华跟随王作英教授学习、研究语音识别时，他给了我几十篇文献。 我印象最深的就是贾里尼克和李开复的文章，它们的核心思想就是隐含马尔可夫模型。复杂的语音识别问题居然能如此简单地被表述、解决，我由衷地感叹数学模型之妙。