How To Record Microphone Audio In Javascript And Submit To Dialogflow?

How can I record audio from the microphone in JavaScript and submit it to DialogFlow, without going through a server?

Solution 1:

There are two parts to this question:

1. How to record microphone audio in a format DialogFlow will understand.
2. How to actually submit that audio to DialogFlow, with proper authentication.

Part 1

For recording microphone audio in a format DialogFlow will understand, I use opus-recorder, then convert the blob it returns using the code below:

functionBlobToDataURL(blob: Blob) {
    returnnewPromise((resolve, reject)=>{
        const reader = newFileReader();
        reader.addEventListener("loadend", e=>resolve(reader.resultasstring));
        reader.readAsDataURL(blob);
    }) asPromise<string>;
}

const micRecorder = newRecorder({
    encoderSampleRate: 16000,
    originalSampleRateOverride: 16000, // necessary due to Google bug? (https://github.com/chris-rudmin/opus-recorder/issues/191#issuecomment-509426093)encoderPath: PATH_TO_ENCODER_WORKER_JS,
});
micRecorder.ondataavailable = async typedArray=>{
    const audioData = newBlob([typedArray], {type: "audio/ogg"});
    const audioData_dataURL = awaitBlobToDataURL(audioData);
    const audioData_str = audioData_dataURL.replace(/^data:.+?base64,/, "");

    // here is where you need part 2, to actually submit the audio to DialogFlow
};
micRecorder.start();

Part 2

To submit the audio-data to DialogFlow, see my answer here: https://stackoverflow.com/a/57857698/2441655