You Write the Melody, Computer Handles Harmony

Ching-Hua Chuan once played in Taiwanese rock bands, but has now co-developed a computer program that generates musical accompaniment. Credit: Ching-Hua Chuan

A schematic showing how the ASSA computer program comes up with accompaniment. Credit: USC Viterbi School of Engineering

It might be better than Guitar Hero. A new computer program can generate spot-on music accompaniment tailored to the unique styles of different bands.

Musicians can usually pluck a new melody on their guitar, but finding the right accompaniment can challenge those without formal music training.

"It's easier to come up with a melody than come up with a series of chords to harmonize that melody," said Ching-Hua Chuan, a former Taiwanese rock guitarist turned computer scientist at Barry University in Miami Shores, Fla.

She had tackled the problem for her Ph.D. dissertation under Elaine Chew, an engineer at the University of Southern California in Los Angeles. Together they developed a program that combines computer learning with music theory.

Chuan chose to train the program on five rock albums from her favorite musicians, including Radiohead, Green Day and Keane. She and Chew tested the program by having it create accompaniment for certain songs, and then compared the results to the actual accompaniment recorded by the artists.

An early test with part of Radiohead's song "Creep" identified the correct melodic notes for chord tones 82 percent of the time. Testing with "High and Dry" produced a 70.5 percent accuracy rate. More recent tests with all five rock albums showed that 75 to 80 percent of generated chords closely resembled the original song accompaniments.

Perhaps the most unusual outcome was that the program, called Automatic Style Specific Accompaniment (ASSA), performed best after analyzing just one song.

"That's really a surprising result, because for most machine learning techniques, we want more data to guarantee better results," Chuan told LiveScience.

Previous programs trained on hundreds of songs and focused on generating musically pleasant accompaniment defined by broad music genres, rather than trying to capture an individual band's style.

"As you add more pieces to the mix, these can dilute the style and wash out the fine details that define what's special about a piece," said Chew, who is herself an accomplished pianist and seemed ideally suited to mentor Chuan's interest in music.  

Success in nailing individual style may come from having a music theory framework for the program. Most programs take a bottom-up approach that starts from scratch without the rules of music theory to go by.

ASSA first uses learning techniques based on previous experience to identify notes that form the spine of the main melody, and builds accompaniment chords around those. It also identifies checkpoints where harmonies are clear at certain points in each song.

The program then applies Neo-Riemannian music theory to create chains of accompaniment chords between the checkpoints, based on the particular smooth music transitions that a musician stylistically prefers.

"The music theoretical knowledge does come from the top-down," Chew noted.

Chuan and Chew eventually want to create a program that can produce an entire song from a hummed melody.

• Video - Computer Generated Songwriting
• Video - Play Pinch Harmonics On Guitar
• Why Do We Love Music?