Talk Title: Advances in Large-Scale Shared Memory Multi-processors
This talk answers serious questions from the Big Data community. Can I run my application in memory over my entire data set rather than distributing it over a cluster? Will the solution scale as my problem size increases or must I rewrite my application? Is there a viable alternative to buying a supercomputer?
These questions arise throughout analytics, data mining, data bases, simulation, and biomedical engineering, including computational genomics . The simplest way to run large applications is to run them on a single computer with a single large memory space and many processors. However, when an application's memory requirements exceed the available memory the application may not run.
The option to Scale Up to ever larger servers or supercomputers is often too expensive, so the industry has evolved to a Scale Out approach involving newer technologies to help users slice their problems into pieces small enough to run on more affordable servers. Users scale out by simply adding more servers. But, there are often significant operational and development costs to do this. One has to write new (distributed) software, which is time consuming and susceptible to bugs. Also, as the schemas change, one may have to incur additional data migration expenses of re-partitionion or (re-sharding) data.
We have developed a radically different approach. We've developed a hyperkernel that offers the best of both worlds: it is both user-friendly like Scale Up and grows at linear cost like Scale Out. Running on our hypervisor is like buying a new, larger computer, but without spending huge amounts of money.
The TidalScale hyperkernel combines multiple servers into a single large scalable virtual supercomputer that can run unmodified guest operating systems like Linux, FreeBSD, and Windows, allowing many processors to operate on what appears to be a large, strongly coherent shared memory.
Reception to immediately follow.
![](/sites/default/files/styles/thumbnail/public/default_images/6KbA9AJs_400x400_0.png?itok=JZ3_TJqR)