[No Inputs Prompt Challenge] [Note Detection Demo] Sheets of Sound - Major Studio Fall Prototype 2

Limitations: John Coltrane's Saxophone, No Inputs.

Controls:  NONE

Saxophone plays the notes in the song automatically, and jumps when there's a leap in the song.

Web ver requires Microphone Access, but DOES NOT use microphone inputs.

The purpose of mic access is to use media nodes in a custom JavaScript plugin, which taps Web Audio API directly, bypassing the GetSpectrumData() Unity function that does not work in WebGL builds.

The reason I used media nodes instead of sth like audio buffer, is because mic audio access is ezier to implement than dynamic audio file reading between JS and Unity, also more robust and efficient without advanced optimization.

Also, despite all the efforts mentioned in the change log, the WebGL ver note detection still produced noticeably worse results than compared to non-Web platforms. If I have to take this further I need to create a C# test harness and "machine-learn" the frequency array generated by the two separate approaches, and also take a college physics course on Fourier transforms. Which would encroach more on the already crazy scope of this mini-prototype.

====================

A prototype for a side-scrolling auto battler based on simple note detection without importing any existing external (to default unity) audio processing libraries. The protagonist is John Coltrane's Selmer Mark VI tenor saxophone, playing notes and leaps according to the music playing from the saxophone.

John Coltrane is known for inventing the modal jazz format and sheets of music technique. Which are horizontal performance styles, and piling large amounts of variations of chords into one piece. (I know nothing about music theory so the explanation might be flawed.)

So I created a horizontal side-scroller where Coltrane's Sax automatically pile sheets of sounds/notes onto the obstacles (not yet implemented), based on a song chosen through note detection.

The current prototype is only tuned when detect six note ranges about every 1/4 of a second, which is not enough sample size to tune simultaneously for not-extremely-similar tracks. So so far it only sort of works for My Favorite Things, stereo ver solo by John Coltrane, supposedly one of his most prominent solo pieces in his modal jazz period.

====================

Change Log:

v0.7.5: Increased the norm factor by  5000 folds, now behaves similar to non-WebGL build. But the 5000 magic number is only found because the linearized amplitude was always around 4 magnitudes smaller than Unity's, and a few tests show that 5000 folds yield the closest results in terms of jumps and notes created.

v0.7: Adjusted norm factor to compensate for hanning window in relation to FFT size.

v0.6: Added Hanning Window.

v0.5.5: Fixed conversion bugs in attempt to fix amplitude mismatch.

v0.5: Turns out JS function returns decibels instead of linear amplitude, fixed that.

v0.4: Added JS plugin that are imported only in Web Builds, which replaces the need of GetSpectrumData() funciton.

v0.3: Fixed web build window size bugs. (No longer necessary since problem is with GetSpectrumData() )