Crowd-Sourced RNA Designs Outclass Computer Algorithms

An energetic group of nonexperts, working through an online interface and receiving feedback from lab experiments, has generated RNA (ribonucleic acid) molecule designs that are consistently more effective than those generated by the best computerized design algorithms.

Moreover, the researchers collected some of the best design rules and practices generated by players of the online EteRNA design challenge, and employing machine learning principles, generated their own automated design algorithm, EteRNABot, which also outperform earlier design algorithms. Although this optimized computer design application is faster than humans, the designs it generates still do not match the quality of those of the online community, which now has more than 130,000 members.

The research was published January 27, 2014, in the Proceedings of the National Academy of Sciences of the United States of America (PNAS) online early edition. “The quality of the designs produced by the online EteRNA community is just amazing and far beyond what any of us anticipated when we began this project three years ago,” said Dr. Adrien Treiulle, an assistant professor of computer science and robotics at Carnegie Mellon University (Pittsburgh, PA, USA), who leads the project with Dr. Rhiju Das, an assistant professor of biochemistry at Stanford University (Stanford, CA, USA; www.stanford.edu), and Jeehyung Lee, a PhD student in computer science at Carnegie Mellon.

“This wouldn’t be possible if EteRNA members were just spitting out designs using online simulation tools,” Dr. Treuille continued. “By actually synthesizing the most promising designs in Das’ lab at Stanford, we’re giving our community feedback about what works and doesn’t work in the physical world. And, as a result, these nonexperts are providing us insight into RNA design that is significantly advancing the science.”

RNA is one of the three macromolecules vital for life, along with DNA and proteins. Long recognized as a messenger for genetic data, RNA also may play a much larger role as a regulator of cells. Understanding RNA design could be helpful for treating or controlling diseases such as HIV, for creating RNA-based sensors or even for building computers out of RNA.

The researchers, in the project, assessed the performance of the EteRNA community, EteRNABot and two cutting-edge RNA design algorithms in generating designs that would cause RNA strands to fold themselves into specific shapes. The computers could generate designs in less than one minute, while most people would take one or two days; synthesizing the molecules to determine the success and quality took a month for each design, so the entire experiment lasted about a year.

Ultimately, Dr. Lee reported, the designs produced by humans had a 99% likelihood of being superior to those of the earlier computer algorithms, whereas EteRNABot produced designs with a 95% probability of outdoing the earlier algorithms. “The quality of the community’s designs is so good that even if you generated thousands of designs with computer algorithms, you’d never find one as good as the community’s,” Mr. Lee said.

When the project began, players were asked to design RNA that folded into specific shapes selected by the Das lab. Due to technologic advances that now enable Dr. Das and his team to synthesize 1,000 design sequences monthly instead of the original 30, EteRNA has become an open research project to which researchers from labs around the world can submit design challenges.

Even though EteRNA players may not be scientifically trained, they nonetheless have instincts that, when reinforced by the lab experiments, can lead to new insights. “Most players didn’t have tactical insights on RNA designs,” Mr. Lee said. “They would just recognize patterns—visual patterns. Scientifically, not all of these rules initially seemed to make sense, but people who were following them did better.”

One design rule generated by the players involves “capping.” RNA consists of long sequences of pairs of nucleotides and typically the simplest way to create a sequence or “stack” that will not tear itself apart when synthesized is to fill it with guanine-cytosine (GC) pairs. However, too many GC pairs can generate some unforeseen shapes when synthesized, “It’s like doing origami with a cardboard box,” as one player noted.

According to Mr. Lee, the players found a solution by putting the GC pairs only at the end of the stack—capping—and filling the rest of the stack with adenine-uracil pairs. The investigators are now looking at expanding its design regimen to include three-dimensional (3D) designs. They also are developing a template that researchers in other fields can use to convert scientific projects into online challenges.

Related Links:
Carnegie Mellon University
Stanford University
EteRNA design challenge