Building ChucK(JS) with Gradle

June 29, 2015

As part of my work to port the ChucK music programming language to JavaScript, via the Emscripten C++ to JavaScript compiler, I’ve implemented a Gradle build script for it (instead of the original Makefile). Aside from my working for Gradle, I think it’s a great choice for this kind of polyglot project, where both native and JavaScript targets are built. ChucK is also highly cross platform (i.e. Linux, OS X, Windows), which makes writing a general build script for it demanding. Gradle is a huge help here, especially after its recently gaining support for C and C++.

Aside from the build script itself, 'build.gradle', I had to include a wrapper script (generated by my global installation of Gradle) 'gradlew' ('gradlew.bat' for Windows), as well as 'gradle/wrapper/gradle-wrapper.jar' and 'gradle/wrapper/gradle-wrapper.properties'.

Configurability

The build has been made user configurable along two axes: Debuggability and Linux sound architecture. The first means that unless -P debug=false is specified on the command line, targets are built for debuggability. Otherwise, targets will be built in optimized mode. The latter means that on Linux, one can choose between three sound architectures: ALSA (-P audioArchitecture=alsa, the default), PulseAudio (-P audioArchitecture=pulse) and JACK (-P audioArchitecture=jack).

Modularization

The Gradle build script has been modularized into plugins for the Lex/Yacc and Emscripten tasks, in order to keep the main script as declarative as possible. Furthermore, the Emscripten plugin is only loaded if Emscripten is detected in the OS environment.

Native Compilation

The native (i.e. non-JavaScript) version of ChucK is simply a console executable, which makes it fairly straightforward to build via Gradle’s component model. Such a target is simply represented by a NativeExecutableSpec in the component model, so for the most part the difficulty lies in determining platform dependent parameters and which C and C++ files to compile. Lex and Yacc tools must also be invoked in order to compile the lexer and compiler components respectively, for which there must be custom build steps (due to lack of direct support in the component model).

To build the ChucK console command ('build/binaries/chuckExecutable/chuck'), simply issue the following command: ./gradlew assemble.

The main build script (build.gradle) looks as follows:

wrapper {
  gradleVersion = "2.5"
}

apply plugin: 'cpp'
apply plugin: 'c'
apply plugin: 'lexAndYacc'

enum Platform {
  OsX('mac os x|darwin|osx'),
  Windows('windows'),
  Linux('linux')
  Platform(String regex) { this.regex = regex }
  String regex
  static Platform current() {
    def osName = System.getProperty('os.name')
     values().find { osName =~ "(?i)${it.regex}" }
   }
}

ext.platform = Platform.current()
def debug = true
if (project.hasProperty('debug')) {
  debug = project.debug.toLowerCase().matches(/^(true|1)$/)
}
if (debug) {
  if (project.platform == Platform.OsX || project.platform == Platform.Linux) {
    ext.compilerFlags = ['-g',]
  }
} else {
  if (project.platform == Platform.OsX || project.platform == Platform.Linux) {
    ext.compilerFlags = ['-O3',]
  }
}

binaries.all {
  // Define toolchain-specific compiler and linker options
  cCompiler.args project.compilerFlags as String[]
  cppCompiler.args project.compilerFlags as String[]

  if (project.platform == Platform.OsX) {
    // TODO: Determine automatically
    cCompiler.args '-DHAVE_CONFIG_H', '-D__MACOSX_CORE__', '-isysroot',
      '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk', \
      '-mmacosx-version-min=10.4'
    cppCompiler.args '-D__MACOSX_CORE__', '-isysroot', \
      '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk', \
      '-mmacosx-version-min=10.4'
    linker.args '-F/System/Library/PrivateFrameworks', '-weak_framework', 'MultitouchSupport', \
      '-framework', 'CoreAudio', '-framework', 'CoreMIDI', '-framework', 'CoreFoundation', '-framework', \
      'IOKit', '-framework', 'Carbon', '-framework', 'AppKit', '-framework', 'Foundation', \
      '-mmacosx-version-min=10.4'
  } else if (project.platform == Platform.Linux) {
    if (!debug) {
      cCompiler.args '-fno-strict-aliasing'
      cppCompiler.args '-fno-strict-aliasing'
    }
    cCompiler.args '-DHAVE_CONFIG_H', '-D__PLATFORM_LINUX__ -D__CK_SNDFILE_NATIVE__'
    cppCompiler.args '-D__PLATFORM_LINUX__ -D__CK_SNDFILE_NATIVE__'

    def audioArch = (project.hasProperty('audioArchitecture')) ? audioArchitecture.toLowerCase() : 'alsa'
    if (audioArch == 'alsa') {
      cCompiler.args '-D__LINUX_ALSA__'
      cppCompiler.args '-D__LINUX_ALSA__'
      linker.args '-lasound'
    } else if (audioArch == 'jack') {
      cCompiler.args '-D__UNIX_JACK__'
      cppCompiler.args '-D__UNIX_JACK__'
      linker.args '-lasound', '-ljack '
    } else if (audioArch == 'pulse') {
      cCompiler.args '-D__LINUX_PULSE__'
      cppCompiler.args '-D__LINUX_PULSE__'
      linker.args '-lasound', '-lpulse-simple', '-lpulse'
    } else {
      throw new Exception("Unknown audioArchitecture option '${audioArchitecture}'")
    }

    linker.args '-lstdc++', '-ldl', '-lm', '-lsndfile', '-lpthread'
  }
}

model {
  components {
    chuck(NativeExecutableSpec) {
      sources.all {
        exportedHeaders {
          srcDir 'src'
          srcDir 'src/lo'
        }
      }
      sources {
        yaccOutput(CSourceSet) {
          generatedBy tasks.yacc
          source.include '*.c'
        }
        lexOutput(CSourceSet) {
          generatedBy tasks.lex
          source.include '*.c'
          lib sources.yaccOutput
        }
        c {
          source {
            srcDir 'src'
            include '*.c'
            include 'lo/*.c'
            if (project.platform != Platform.Windows) {
              exclude 'chuck_win32.c'
            }
            if (project.platform != Platform.OsX) {
              exclude 'util_sndfile.c'
            }
          }
          lib sources.lexOutput
          lib sources.yaccOutput
        }
        cpp {
          source {
            srcDir 'src'
            include '*.cpp'
            include 'RtAudio/*.cpp'
            exclude 'rtaudio_c.cpp'
            exclude 'chuck_js.cpp'
            exclude 'digiio_webaudio.cpp'
          }
        }
      }
    }
  }
}

if (System.getenv("EMSCRIPTEN")?.trim()) {
  logger.info("Emscripten detected - Configuring targets")
  apply plugin: 'emscripten'
} else {
  logger.info("Emscripten not detected - Not configuring targets")
}

Lex and Yacc Plugin

The tasks for compiling the lexer and compiler components are modularized into a separate plugin, 'buildSrc/src/main/groovy/LexAndYaccPlugin.groovy'. To make the plugin importable by the build script, it has an accompanying metadata file 'buildSrc/src/main/resources/META-INF/gradle-plugins/lexAndYacc.properties', with the following content:

implementation-class=LexAndYaccPlugin

The plugin itself is implemented in the following way:

import org.gradle.api.*
import org.gradle.api.file.*
import org.gradle.api.tasks.*

class LexCompiler extends DefaultTask {
  @InputFiles FileCollection lexFiles
  @OutputDirectory File sourceDir
  @OutputDirectory File headerDir

  @TaskAction
  void processLexFiles() {
    lexFiles.each { file ->
      project.exec {
        commandLine = ['flex', '-o', new File(sourceDir, file.name - '.lex' + '.yy.c').absolutePath,
          file.absolutePath]
      }
    }
  }
}

class YaccCompiler extends DefaultTask {
  @InputFiles FileCollection yaccFiles
  @OutputDirectory File sourceDir
  @OutputDirectory File headerDir

  @TaskAction
  void processYaccFiles() {
    yaccFiles.each { file ->
      project.exec {
        commandLine = ['bison', '-dv', '-o', new File(sourceDir, file.name - '.y' + '.tab.c'),
          file.absolutePath]
      }
    }
  }
}

class LexAndYaccPlugin implements Plugin {
  void apply(Project project) {
    project.task('yacc', type: YaccCompiler) {
      sourceDir = project.file("${project.buildDir}/src/generated/yacc")
      headerDir = project.file("${project.buildDir}/src/generated/yacc")
      yaccFiles = project.files('src/chuck.y')
    }

    project.task('lex', type: LexCompiler) {
      sourceDir = project.file("${project.buildDir}/src/generated/lex")
      headerDir = project.file("${project.buildDir}/src/generated/lex")
      lexFiles = project.files('src/chuck.lex')
    }
  }
}

Building JavaScript Library via Emscripten

Here comes the interesting part (IMHO). To turn ChucK into a JavaScript library which can be run right inside your Web browser, we employ the wizardry of the Emscripten C++ to JavaScript compiler. This means that for 99% of the code, there’s a straight translation from the C/C++ source code to the JavaScript equivalent. Only a touch of adapter code is required to make ChucK interface with the Web Audio API.

Writing the build logic for the JavaScript target is a bit more challenging though, mainly because there is no direct support for Emscripten in Gradle. Instead, you have to write several tasks and task types, in order to compile the various sources through Emscripten and finally to link them together into a JavaScript library (chuck.js).

To compile the ChucK JavaScript library ('build/js/chuck.js'), first activate the Emscripten SDK in your shell (would be sweet if Gradle provisioned it for you wouldn’t it?), and then issue the command ./gradlew emscripten.

As described earlier, the Emscripten part of the build logic is implemented as a separate plugin. It is made importable through a metadata file, 'buildSrc/src/main/resources/META-INF/gradle-plugins/emscripten.properties', with the following content:

implementation-class=EmscriptenPlugin

The Emscripten plugin itself, 'buildSrc/src/main/groovy/EmscriptenPlugin.groovy', is implemented as follows:

import org.gradle.api.*
import org.gradle.api.plugins.*
import org.gradle.api.file.*
import org.gradle.api.tasks.*
import org.gradle.api.tasks.incremental.IncrementalTaskInputs

class EmscriptenBaseTask extends DefaultTask {
  def emccPath = new File(System.getenv("EMSCRIPTEN"), 'emcc').absolutePath
}

class EmscriptenCompiler extends EmscriptenBaseTask {
  @InputFiles FileCollection emscriptenFiles
  @OutputDirectory File outputDirectory

  def replaceExtension(fileName) {
    return fileName.replaceFirst(/\.(c|cpp)$/, '.o')
  }

  @TaskAction
  void processEmscriptenFiles(IncrementalTaskInputs inputs) {
    inputs.outOfDate { change ->
      project.exec {
        // TODO: Don't hardcode YACC header dir
        commandLine = [emccPath, '-Isrc', '-Isrc/lo', '-Ibuild/src/generated/yacc', '-o',
            new File(outputDirectory, replaceExtension(change.file.name)).absolutePath] +
        project.compilerFlags + [change.file.absolutePath]
      }
    }

    inputs.removed { change ->
      new File(outputDirectory, replaceExtension(change.file.name)).delete()
    }
  }
}

class EmscriptenLinker extends EmscriptenBaseTask {
  @InputFiles FileCollection emscriptenFiles
  @OutputDirectory File outputDirectory

  @TaskAction
  void processEmscriptenFiles() {
    project.exec {
      // TODO: Make -g and SAFE_HEAP configurable
      commandLine = [emccPath, '-g', '-s', 'EXPORTED_FUNCTIONS=["_executeCode"]', '--js-library',
        'src/emscripten/libraries/webaudio.js', '-s', 'SAFE_HEAP=1', '-s', 'DEMANGLE_SUPPORT=1', '-o',
        new File(outputDirectory, 'chuck.js').absolutePath] + emscriptenFiles.collect { it.absolutePath }
    }
  }
}

class EmscriptenPlugin implements Plugin {
  void apply(Project project) {
    project.task('emscriptenYacc', type: EmscriptenCompiler) {
      outputDirectory = project.file("${project.buildDir}/emscripten")
      emscriptenFiles = project.fileTree(project.tasks.yacc.sourceDir)
        .include('chuck.tab.c')
      dependsOn(project.tasks.yacc)
    }

    project.task('emscriptenLex', type: EmscriptenCompiler) {
      outputDirectory = project.file("${project.buildDir}/emscripten")
      emscriptenFiles = project.fileTree(project.tasks.lex.sourceDir)
        .include('chuck.yy.c')
      dependsOn(project.tasks.lex)
      dependsOn(project.tasks.yacc)
    }

    project.task('emscriptenCompile', type: EmscriptenCompiler) {
      outputDirectory = project.file("${project.buildDir}/emscripten")
      emscriptenFiles = project.fileTree(dir: 'src')
        .include('*.c')
        .include('*.cpp')
        .exclude('rtaudio_c.cpp')
        .exclude('digiio_rtaudio.cpp')
        .exclude('util_sndfile.c')
        .exclude('chuck_win32.c')

      dependsOn(project.tasks.yacc)
    }

    project.task('emscripten', type: EmscriptenLinker) {
      outputDirectory = project.file("${project.buildDir}/js")
      emscriptenFiles = project.tasks.emscriptenYacc.outputs.files.asFileTree + \
        project.tasks.emscriptenLex.outputs.files.asFileTree + \
        project.tasks.emscriptenCompile.outputs.files.asFileTree
    }
  }
}

To see the whole build script, go here.

In a future blog post, I will detail how to build also the ChucKJS example HTML pages with Gradle, replacing Grunt in the process.