Dec 08, 2011 6:54 PM GMT
The Big Data research challenge is to develop technology that can obtain timely and cost-effective answers to Big Data questions. A Berkeley team of eight faculty members and 40 Ph.D. students is rising to that challenge via three initiatives: inventing algorithms based on statistical machine learning; harnessing many machines in the cloud; and developing crowd-sourcing techniques to get people to help answer questions that prove too hard for our algorithms and machines.
Algorithms, machines and people gave our new lab its name: the AMP Lab.
AMP technology could help the war on cancer. It needs new algorithms to find those needles in haystacks. To process genome data faster and more cheaply, the war needs new infrastructure to use many machines in the cloud simultaneously. And it needs to be able to engage the wisdom of the crowd when the problems of cancer genome discovery and diagnosis are beyond our algorithms and machines.
It may have been true once that expertise in computer science was needed only by computer scientists. But Big Data has shown us that’s no longer the case. It is entirely possible that we have the skill sets needed now to fight cancer and to advance sciences in myriad other ways.
The night after we made that argument, I awoke in the middle of the night with this question etched into my mind: Given that millions of people do have and will get cancer, if there is a chance that computer scientists may have the best skill set to fight cancer today, as moral people aren’t we obligated to try?