Preliminaries

MaryTTS

phone	pos_in_syl	accented	ph_cplace
h	0	0	g
@	1	0	0
l	0	1	a
@U	1	1	0
w	0	1	l
r=	1	1	0
l	2	1	a
d	3	1	a
_	0	0	0

Build with Gradle

./gradlew legacyInit
./gradlew build

Pitchmarking

Using Praat

./gradlew praatPitchmarker

input: wav/*.wav
output: pm/*.pm

MCEP extraction

Using ch_track from EST

./gradlew mcepExtractor

input: wav/*.wav
output: mcep/*.mcep

G2P and labeling

Predict phone sequence from text using MaryTTS

./gradlew generateAllophones

input: text/*.txt
output: prompt_allophones/*.xml

Check phonetic alignment

./gradlew legacyTranscriptionAligner

input: prompt_allophones/*.xml, lab/*.lab
output: allophones/*.xml

Unit features

Compute and assign feature vector to each unit using MaryTTS

./gradlew legacyPhoneUnitFeatureComputer legacyHalfPhoneUnitFeatureComputer

input: allophones/*.xml, mary/features.txt
output: phonefeatures/*.pfeats, halfphonefeatures/*.hpfeats

Data files

Compile “timeline” files for audio, utterances, and acoustic features

./gradlew legacyWaveTimelineMaker legacyBasenameTimelineMaker legacyMCepTimelineMaker

input: wav/*.wav, pm/*.pm, mcep/*.mcep
output: mary/timeline_waveforms.mry, mary/timeline_basenames.mry, mary/timeline_mcep.mry

These contain the actual data from the wav and mcep files, in pitch-synchronous “datagram” packets.

Acoustic models

Phone-level and halfphone-level unit and features files

./gradlew legacyPhoneUnitfileWriter legacyHalfPhoneUnitfileWriter legacyPhoneFeatureFileWriter legacyHalfPhoneFeatureFileWriter

input: pm/*.pm, phonelab/*.lab, phonefeatures/*.pfeats, halfphonelab/*.hplab
output: mary/phoneUnits.mry, mary/halfphoneUnits.mry, mary/phoneFeatures.mry, mary/phoneUnitFeatureDefinition.txt, mary/halfphoneFeatures.mry, mary/halfphoneUnitFeatureDefinition.txt

CARTs for Duration and F0

Using wagon from EST

./gradlew legacyDurationCARTTrainer legacyF0CARTTrainer

input: mary/phoneUnits.mry, mary/phoneFeatures.mry, mary/timeline_waveforms.mry
output: mary/dur.tree, mary/f0.left.tree, mary/f0.mid.tree, mary/f0.right.tree

Distributable voice package

Ready for deployment in MaryTTS installation

./gradlew assemble

input: mary/cart.mry, featureSequence.txt, mary/dur.tree, mary/f0.left.tree, mary/f0.mid.tree, mary/f0.right.tree, mary/halfphoneFeatures_ac.mry, mary/joinCostFeatures.mry, mary/joinCostWeights.txt, mary/halfphoneUnits.mry, mary/timeline_basenames.mry, mary/timeline_waveforms.mry
output: my_voice.zip, my_voice-component.xml

Simple Gradle build script

build.gradle

task foo {
  doLast {
    println "Doing stuff."
  }
}

task bar {
  dependsOn foo
  doLast {
    println "Doing more stuff."
  }
}

00:00-00:00

Custom task class

buildSrc/src/main/groovy/DoStuff.groovy

class DoStuff extends DefaultTask {

    @InputFile
    final RegularFileProperty inputFile = newInputFile()

    @OutputFile
    final RegularFileProperty outputFile = newOutputFile()

    @TaskAction
    void doStuff() {
        // open output file for writing
        outputFile.get().asFile.withWriter { writer ->
            // iterate over lines in input file
            inputFile.get().asFile.eachLine { line ->
                // write line contents to output, twice
                writer.println line * 2
            }
        }
    }
}

Further reading

Lazy task input/output

src/foo.txt

foo
bar
baz

build.gradle

task foo(type: DoStuff) {
    inputFile = file('src/foo.txt')
    outputFile = layout.buildDirectory.file('bar.txt')
}

task bar(type: DoStuff) {
    inputFile = foo.outputFile
    outputFile = layout.buildDirectory.file('baz.txt')
}

00:00-00:00

Custom plugin

buildSrc/src/main/groovy/DoPlugin.groovy

class DoPlugin implements Plugin<Project> {

    @Override
    void apply(Project project) {
        project.task('foo', type: DoStuff) {
            outputFile = project.layout.buildDirectory.file('bar.txt')
        }

        project.task('bar', type: DoStuff) {
            inputFile = project.foo.outputFile
            outputFile = project.layout.buildDirectory.file('baz.txt')
        }
    }
}

build.gradle

apply plugin: DoPlugin

foo.inputFile = file('src/foo.txt')

Further reading

Writing Custom Plugins

Dependencies

build.gradle

repositories {
    ivy {
        url 'https://catalog.ldc.upenn.edu/docs'
        layout 'pattern', {
            artifact 'LDC93S1/[module].[ext]'
        }
    }
}

configurations {
    data
}

dependencies {
    data group: 'edu.upenn.ldc.timit', name: 'PROMPTS', version: '1988-10-31', ext: 'TXT'
}

tasks.register 'processPrompts', ProcessPrompts, {
    srcFiles = files(configurations.data)
    destDir = layout.buildDirectory.dir('text')
}

Further reading

Declaring Dependencies

Custom dependency processing class

buildSrc/src/main/groovy/ProcessPrompts.groovy

import org.gradle.api.DefaultTask
import org.gradle.api.file.*
import org.gradle.api.tasks.*

class ProcessPrompts extends DefaultTask {

    @InputFiles
    FileCollection srcFiles = project.files()

    @OutputDirectory
    final DirectoryProperty destDir = newOutputDirectory()

    @TaskAction
    void process() {
        project.copy {
            from srcFiles
            into destDir.get().asFile
            eachFile { fileDetails ->
                fileDetails.file.eachLine { line ->
                    if (line.startsWith(';'))
                        return
                    (line =~ /(.+) \((.+)\)/).each { all, prompt, code ->
                        destDir.file("${code}.txt").get().asFile.withWriter { writer ->
                            writer.println prompt
                        }
                    }
                }
                fileDetails.exclude()
            }
        }
    }
}

Further reading

Voicebuilding for Text-to-Speech Synthesis