Cerebras

Cerebras Systems Inc. is an American artificial intelligence company with offices in Sunnyvale, San Diego, Toronto, Tokyo[2] and Bangalore.[3][4] Cerebras builds computer systems for complex artificial intelligence deep learning applications.[5]

Cerebras Systems Inc.
TypePrivate
Industry
Founded2015 (2015)
Founders
  • Andrew Feldman
  • Gary Lauterbach
  • Michael James
  • Sean Lie
  • Jean-Philippe Fricker
Headquarters
Sunnyvale, California
,
US
Key people
Andrew Feldman (CEO)
ProductsWafer Scale Engine
Number of employees
335 (2023)
Websitewww.cerebras.net
Footnotes / references
[1]

History

Cerebras was founded in 2015 by Andrew Feldman, Gary Lauterbach, Michael James, Sean Lie and Jean-Philippe Fricker.[6] These five founders worked together at SeaMicro, which was started in 2007 by Feldman and Lauterbach and was later sold to AMD in 2012 for $334 million.[7][8]

On August 19, 2019, Cerebras announced its Wafer-Scale Engine (WSE).[9][10][11]

In 2020, the company announced an office in Japan and partnership with Tokyo Electron Devices.[12]

In April 2021, Cerebras announced the CS-2 based on the company's Wafer Scale Engine Two (WSE-2), which has 850,000 cores.[2] In August 2021, the company announced its brain-scale technology that can run a neural network with over 120 trillion connections.[13]

In August 2022, Cerebras was honored by the Computer History Museum in Mountain View, California. The museum added to its permanent collection and unveiled a new display featuring the WSE-2—the biggest computer chip made so far—marking an "epochal" achievement in the history of fabricating transistors as an integrated part.[14][15]

In August 2022, Cerebras announced the opening of a new office in Bangalore, India.[3][4]

Funding

Cerebras secured $27 million in series A funding led by Benchmark, Foundation Capital and Eclipse Ventures in May 2016.[16][6] In December 2016, series B funding was led by Coatue Management, followed in January 2017 with series C funding led by VY Capital.[6]

The company closed its series D round with $88 million, making the company a unicorn. Investors in this round included Altimeter, VY Capital, Coatue, Foundation Capital, Benchmark, and Eclipse, in November 2018.[17][18] The following November, Cerebras closed its series E round with over $270 million for a valuation of $2.4 billion.[19] In November 2021, Cerebras announced that it had raised an additional $250 million in Series F funding, valuing the company at over $4 billion. The Series F financing round was led by Alpha Wave Ventures and Abu Dhabi Growth Fund (ADG).[20]

To date, the company has raised $720 million in financing.[20][21]

Technology

The Cerebras Wafer Scale Engine (WSE) is a single, wafer-scale integrated circuit processor that includes compute, memory, and interconnect fabric. Scheduling uses a dataflow architecture.[22][23]

The WSE-1 powers the Cerebras CS-1, the firm's first-generation AI computer.[24] It is a 19-inch rack-mounted appliance designed for AI training and inference workloads in a datacenter.[10] The CS-1 includes one WSE primary processor with 400,000 processing cores, and twelve 100 Gigabit Ethernet connections for data input/output.[25][10] The WSE-1 has 1.2 trillion transistors, 400,000 compute cores and 18 gigabytes of memory.[9][10][11]

In April 2021, Cerebras announced the CS-2 AI system based on the 2nd-generation Wafer Scale Engine (WSE-2), manufactured by the 7 nm process of Taiwan Semiconductor Manufacturing Company (TSMC).[2] It is 26 inches tall and fits in one-third of a standard data center rack.[26][2] The WSE-2 has 850,000 cores and 2.6 trillion transistors.[26][27] The WSE-2 expanded on-chip static random-access memory (SRAM) to 40 gigabytes, memory bandwidth to 20 petabytes per second and total fabric bandwidth to 220 petabits per second.[28][29]

In August 2021, the company announced a system which connects multiple integrated circuits (commonly called chips) into a neural network with many connections.[13] It enables one system to support AI models with more than 120 trillion parameters.[30]

In June 2022, Cerebras set a record for the largest AI models ever trained on one device.[31] Cerebras said that for the first time ever, one CS-2 system with one Cerebras wafer can train models with up to 20 billion parameters.[32] The Cerebras CS-2 system can train multibillion-parameter natural language processing (NLP) models including GPT-3XL 1.3 billion models, GPT-J-6B, GPT-3 13B, and GPT-NeoX 20B with reduced software complexity and infrastructure.[32][31]

In August 2022, Cerebras announced that its customers can now train Transformer-style natural language AI models with 20x longer sequences than possible using traditional computer hardware, which is expected to lead to breakthroughs in natural language processing (NLP), especially for pharmaceuticals and life sciences.[33]

In September 2022, Cerebras announced that it can patch its chips together to create what would be the largest-ever computing cluster for AI computing.[34] A Wafer-Scale Cluster can connect up to 192 CS-2 AI systems into a cluster, while a cluster of 16 CS-2 AI systems can create a computing system with 13.6 million cores for natural language processing.[34] The key to the new Cerebras Wafer-Scale Cluster is the exclusive use of data parallelism to train, which is the preferred approach for all AI work.[35]

In November 2022, Cerebras unveiled its latest supercomputer, Andromeda, which combines 16 WSE-2 chips into one cluster with 13.5 million AI-optimized cores, delivering up to 1 exaFLOPS of AI computing power, or at least one quintillion (1018) operations per second.[36][37] The entire system uses 500 kilowatts, which is far less power than somewhat-comparable GPU-accelerated supercomputers.[36]

In November 2022, Cerebras announced its partnership with Cirrascale Cloud Services to provide a flat-rate "pay-per-model" compute time for its Cerebras AI Model Studio. Pricing ranges from $2,500 for training "a 1.3-billion-parameter model of GPT-3 in 10 hours" to $2.5 million for training "70-billion-parameter version in 85 days". The service is said to reduce the cost—compared to the similar cloud services on the market—by half while increasing speed up to eight times faster.[38]

Cerebras-GPT Series

In March 2023, Cerebras released its Cerebras-GPT models: a series of 7 open source LLMs with 111M, 256M, 590M, 1.3B, 2.7B, 6.7B, and 13B parameters respectively.[39] This was reported to be the first family of GPT models that are compute-efficient at every model size; earlier open GPT models are trained on a fixed number of data tokens. However, other AI researchers like Stella Biderman of EleutherAI have noted that the practical use of such compute-efficient models is limited, as they underperform significantly on benchmark tests relative to other open-source models with similar parameter sizes, such as the Pythia suite.[40]

Deployments

Customers are reportedly using Cerebras technologies in the pharmaceutical, life sciences, and energy sectors.[41][42]

In 2020, GlaxoSmithKline (GSK) began using the Cerebras CS-1 AI system in their London AI hub, for neural network models to accelerate genetic and genomic research and reduce the time taken in drug discovery.[43] The GSK research team was able to increase the complexity of the encoder models they could generate, while reducing training time.[44] Other pharmaceutical industry customers include AstraZeneca, who was able to reduce training time from two weeks on a cluster of GPUs to two days using the Cerebras CS-1 system.[45] GSK and Cerebras recently co-published research in December 2021 on epigenomic language models.

Argonne National Laboratory has been using the CS-2/CS-1 since 2020 in COVID-19 research and cancer tumor research based on the world’s largest cancer treatment database.[46] A series of models running on the CS-1 to predict cancer drug response to tumors achieved speed-ups of many hundreds of times on the CS-1 compared to their GPU baselines.[41]

Cerebras and the National Energy Technology Laboratory (NETL) demonstrated record-breaking performance of Cerebras' CS-1 system on a scientific compute workload in November 2020. The CS-1 was 200 times faster than the Joule Supercomputer on the key workload of Computational Fluid Dynamics.[47]

The Lawrence Livermore National Lab’s Lassen supercomputer incorporated the CS-1 in both classified and non-classified areas for physics simulations.[48] The Pittsburgh Supercomputing Center (PSC) has also incorporated the CS-1 in their Neocortex supercomputer for dual HPC and AI workloads.[49] EPCC, the supercomputing center of the University of Edinburgh, has also deployed a CS-1 system for AI-based research.[50]

In August 2021, Cerebras announced a partnership with Peptilogics on the development of AI for peptide therapeutics.[51]

In March 2022, Cerebras announced that the Company deployed its CS-2 system in the Houston facilities of TotalEnergies, its first publicly disclosed customer in the energy sector.[42] Cerebras also announced that it has deployed a CS-2 system at nference, a startup that uses natural language processing to analyze massive amounts of biomedical data. The CS-2 will be used to train transformer models that are designed to process information from piles of unstructured medical data to provide fresh insights to doctors and improve patient recovery and treatment.[52]

In May 2022, Cerebras announced that the National Center for Supercomputing Applications (NCSA) has deployed the Cerebras CS-2 system in their HOLL-I supercomputer. [53] They also announced that the Leibniz Supercomputing Centre (LRZ) in Germany plans to deploy a new supercomputer featuring the CS-2 system along with the HPE Superdome Flex server.[54] The new supercomputing system is expected to be delivered to LRZ this summer. This will be the first CS-2 system deployment in Europe.[54]

In October 2022, it was announced that the U.S. National Nuclear Security Administration would sponsor a study to investigate using Cerebras' CS-2 in nuclear stockpile stewardship computing.[55][56] The multi-year contract will be executed through Sandia National Laboratories, Lawrence Livermore National Lab, and Los Alamos National Laboratory.[55]

In November 2022, Cerebras and the National Energy Technology Laboratory (NETL) saw record-breaking performance on the scientific compute workload of forming and solving field equations. Cerebras demonstrated that its CS-2 system was as much as 470 times faster than NETL's Joule Supercomputer in field equation modeling.[57]

The 2022 Gordon Bell Special Prize Winner for HPC-Based COVID-19 Research, which honors outstanding research achievement towards the understanding of the COVID-19 pandemic through the use of high-performance computing, used Cerebras' CS-2 system to conduct this award-winning research to transform large language models to analyze COVID-19 variants. The paper was authored by a 34-person team from Argonne National Laboratory, California Institute of Technology, Harvard University, Northern Illinois University, Technical University of Munich, University of Chicago, University of Illinois Chicago, Nvidia, and Cerebras. ANL noted that using the CS-2 Wafer-Scale Engine cluster, the team was able to achieve convergence when training on the full SARS-CoV-2 genomes in less than a day.[58][59]

In November 2022, Cerebras announced a partnership with AI content platform Jasper AI to help accelerate the adoption and accuracy of generative AI. Jasper will use Cerebras' Andromeda AI supercomputer to train its computationally-intensive models in a "fraction of the time."[60]

In February 2023, Cerebras announced that it had carried out, for the first time ever, the simulation of a high-resolution convection workload at near real-time rates using the WSE Field-equation application programming interface developed by the National Energy Technology Laboratory and carried out by the Neocortex AI supercomputer, powered by multiple CS-2 AI systems, at the Pittsburgh Supercomputing Center.[61] Cerebras said the WFA API powered by its CS-2 system simulated Rayleigh-Bénard convection about 470 times faster than what was possible on NETL’s existing Joule Supercomputer, which is powered by traditional GPUs.[61]

Cerebras partnered with Emirati technology group G42 to deploy its AI supercomputers to create chatbots and to analyze genomic and preventive care data. In July 2023, G42 agreed to pay around $100 million to purchase the first of potentially nine supercomputers from Cerebras, each of which capable of 4 exaflops of compute.[62][63][64] In August 2023, Cerebras, the Mohamed bin Zayed University of Artificial Intelligence and G42 subsidiary Inception launched Jais, a large language model.[65]

See also

References
  1. Takahashi, Dean (20 July 2023). "Cerebras unveils world's largest AI training supercomputer with 54M cores". VentureBeat.
  2. "Cerebras launches new AI supercomputing processor with 2.6 trillion transistors". VentureBeat. 2021-04-20. Retrieved 2021-04-30.
  3. "Cerebras Systems Accelerates Global Growth with New India Office". finance.yahoo.com. 30 August 2022. Retrieved 2022-08-30.
  4. Jolly, Andrew. "Cerebras Systems Opens New India Office". HPCwire. Retrieved 2022-08-30.
  5. "Cerebras Systems deploys the 'world's fastest AI computer' at Argonne National Lab". VentureBeat. 2019-11-19. Retrieved 2021-04-30.
  6. Tilley, Aaron. "AI Chip Boom: This Stealthy AI Hardware Startup Is Worth Almost A Billion". Forbes. Retrieved 2021-04-30.
  7. Hardy, Quentin (2012-02-29). "A.M.D. Buying SeaMicro for $334 Million". Bits Blog. Retrieved 2021-04-30.
  8. "How Google Spawned The 384-Chip Server". Wired. ISSN 1059-1028. Retrieved 2021-04-30.
  9. Metz, Cade (2019-08-19). "To Power A.I., Start-Up Creates a Giant Computer Chip". The New York Times. ISSN 0362-4331. Retrieved 2021-04-30.
  10. "The Cerebras CS-1 computes deep learning AI problems by being bigger, bigger, and bigger than any other chip". TechCrunch. 19 November 2019. Retrieved 2021-04-30.
  11. "Full Page Reload". IEEE Spectrum: Technology, Engineering, and Science News. Retrieved 2021-04-30.
  12. Cerebras Systems. "Cerebras Systems Expands Global Footprint with New Offices in Tokyo, Japan, and Toronto, Canada". Press Release. Retrieved August 13, 2021.
  13. "Cerebras' Tech Trains "Brain-Scale" AIs". IEEE Spectrum. 2021-08-24. Retrieved 2021-09-22.
  14. "AI startup Cerebras celebrated for chip triumph where others tried and failed". ZDNet. Retrieved 2022-08-04.
  15. "The Biggest Chip In the World". CHM. 2022-08-03. Retrieved 2022-08-04.
  16. "A stealthy startup called Cerebras raised around $25 million to build deep learning hardware". TechCrunch. 19 December 2016. Retrieved 2021-04-30.
  17. Martin, Dylan (2019-11-27). "AI Chip Startup Cerebras Reveals 'World's Fastest AI Supercomputer'". CRN. Retrieved 2021-04-30.
  18. Strategy, Moor Insights and. "Cerebras Unveils AI Supercomputer-On-A-Chip". Forbes. Retrieved 2021-04-30.
  19. "Cerebras Crams More Compute Into Second-Gen 'Dinner Plate Sized' Chip". EE Times. Retrieved 2021-05-12.
  20. "Cerebras Systems Raises $250M in Funding for Over $4B Valuation to Advance the Future of AI Compute". HPCwire. Retrieved 2021-11-10.
  21. "AI chip startup Cerebras Systems raises $250 million in funding". Reuters. 2021-11-10. Retrieved 2021-11-10.
  22. https://web.archive.org/web/20210427214147/https://www.hotchips.org/assets/program/conference/day2/HotChips2020_ML_Training_Cerebras_v03.pdf
  23. https://hc32.hotchips.org/assets/program/tutorials/HC2020.Cerebras.NataliaVassilieva.v02.pdf
  24. Moore, Samuel K. (1 January 2020). "Cerebras's Giant Chip Will Smash Deep Learning's Speed Barrier". IEEE Spectrum. Retrieved 2021-04-30.
  25. "Neocortex Will Be First-of-Its-Kind 800,000-Core AI Supercomputer". HPCwire. 2020-06-09. Retrieved 2021-04-30.
  26. Ray, Tiernan (April 20, 2021). "Cerebras continues 'absolute domination' of high-end compute, it says, with world's hugest chip two-dot-oh". ZDNet. Retrieved August 13, 2021.
  27. Knight, Will (August 24, 2021). "A New Chip Cluster Will Make Massive AI Models Possible". Wired. ISSN 1059-1028. Retrieved 2021-08-25.
  28. "Cerebras Systems Smashes the 2.5 Trillion Transistor Mark with New Second Generation Wafer Scale Engine". Bloomberg. Retrieved 2021-06-02.
  29. Cutress, Ian. "Cerebras Unveils Wafer Scale Engine Two (WSE2): 2.6 Trillion Transistors, 100% Yield". Anandtech.com. Retrieved 2021-06-03.
  30. Khalili, Joel (25 August 2021). "The world's largest chip is creating AI networks larger than the human brain". TechRadar. Retrieved 2021-09-22.
  31. Francisco Pires (2022-06-22). "Cerebras Slays GPUs, Breaks Record for Largest AI Models Trained on a Single Device". Tom's Hardware. Retrieved 2022-06-22.
  32. Takahashi, Dean (2022-06-22). "Cerebras Systems sets record for largest AI models ever trained on one device". VentureBeat. Retrieved 2022-06-22.
  33. Jolly, Andrew. "Cerebras Announces New Capability for Training NLP Models". HPCwire. Retrieved 2022-09-01.
  34. Shah, Agam (2022-09-14). "Cerebras Proposes AI Megacluster with Billions of AI Compute Cores". HPCwire. Retrieved 2022-09-14.
  35. Freund, Karl. "New Cerebras Wafer-Scale Cluster Eliminates Months Of Painstaking Work To Build Massive Intelligence". Forbes. Retrieved 2022-09-15.
  36. Paul Alcorn (2022-11-14). "Cerebras Reveals Andromeda, a 13.5 Million Core AI Supercomputer". Tom's Hardware. Retrieved 2022-11-18.
  37. Lee, Jane Lanhee (2022-11-14). "Silicon Valley chip startup Cerebras unveils AI supercomputer". Reuters. Retrieved 2022-11-18.
  38. Ray, Tiernan (2022-11-29). "AI challenger Cerebras unveils 'pay-per-model' AI cloud service with Cirrascale, Jasper". ZDNet.
  39. "Cerebras Systems Releases Seven New GPT Models Trained on CS-2 Wafer-Scale Systems" (Press release). 28 March 2023.
  40. https://twitter.com/BlancheMinerva/status/1640768381719592960
  41. "LLNL, ANL and GSK Provide Early Glimpse into Cerebras AI System Performance". HPCwire. 2020-10-13. Retrieved 2021-06-03.
  42. "Cerebras Systems Supplies 2nd-Gen AI System to TotalEnergies". EnterpriseAI. 2022-03-03. Retrieved 2022-03-04.
  43. Ray, Tiernan (September 5, 2020). "Glaxo's biology research with novel Cerebras machine shows hardware may change how AI is done". ZDNet. Retrieved August 13, 2021.
  44. "Cerebras debuts new 2.6 trillion transistor wafer scale chip for AI". www.datacenterdynamics.com. Retrieved 2021-06-17.
  45. Hansen, Lars Lynne (2021-04-26). "Accelerating Drug Discovery Research with New AI Models: a look at the AstraZeneca Cerebras…". Medium. Retrieved 2021-06-03.
  46. Shah, Agam (2020-05-06). "National Lab Taps AI Machine With Massive Chip to Fight Coronavirus". Wall Street Journal. ISSN 0099-9660. Retrieved 2021-06-03.
  47. "Cerebras Systems and NETL Set New Compute Milestone". HPCwire. Retrieved 2022-03-04.
  48. "Cerebras puts 'world's largest computer chip' in Lassen supercomputer". VentureBeat. 2020-08-19. Retrieved 2021-06-03.
  49. Hemsoth, Nicole (2021-03-30). "Neocortex Supercomputer to Put Cerebras CS-1 to the Test". The Next Platform. Retrieved 2022-03-04.
  50. Comment, Dan Swinhoe. "EPCC chooses Cerebras' massive chip for new supercomputer". www.datacenterdynamics.com. Retrieved 2022-03-04.
  51. "Peptilogics and Cerebras Systems Partner on AI Solutions to Advance Peptide Therapeutics". HPCwire. Retrieved 2021-09-22.
  52. "Cerebras brings CS-2 system to data analysis biz nference". www.theregister.com. Retrieved 2022-03-15.
  53. "NCSA Deploys Cerebras CS-2 in New HOLL-I Supercomputer for Large-Scale AI". HPCwire. Retrieved 2022-06-03.
  54. Comment, Sebastian Moss. "Leibniz Supercomputing Centre to deploy HPE-Cerebras supercomputer". www.datacenterdynamics.com. Retrieved 2022-06-03.
  55. "NNSA Taps 3 Federal Labs to Research Applications of Cerebras Systems Tech - ExecutiveBiz". blog.executivebiz.com. 2022-10-18. Retrieved 2022-11-18.
  56. Mann, Tobias. "DoE to trial Cerebras AI compute in nuclear weapon sims". www.theregister.com. Retrieved 2022-11-18.
  57. "Cerebras and National Energy Tech Lab Set New Milestones for High-Performance, Energy-Efficient Field Equation Modeling Using Simple Python Interface". HPCwire. Retrieved 2022-11-18.
  58. Peckham, Oliver (2022-11-17). "Gordon Bell Nominee Used LLMs, HPC, Cerebras CS-2 to Predict Covid Variants". HPCwire. Retrieved 2022-11-23.
  59. Peckham, Oliver (2022-11-17). "Gordon Bell Special Prize Goes to LLM-Based Covid Variant Prediction". HPCwire. Retrieved 2022-11-23.
  60. "Cerebras unveils new partnerships for LLM and generative AI tools". VentureBeat. 2022-11-29. Retrieved 2023-03-13.
  61. "AI chip startup Cerebras Systems announces pioneering simulation of computational fluid dynamics". SiliconANGLE. 2023-02-07. Retrieved 2023-03-13.
  62. Nellis, Stephen; Hu, Krystal (20 July 2023). "Cerebras Systems signs $100 mln AI supercomputer deal with UAE's G42". Reuters.
  63. Lu, Yiwen (20 July 2023). "An A.I. Supercomputer Whirs to Life, Powered by Giant Computer Chips". The New York Times.
  64. Moore, Samuel K. (20 July 2023). "Cerebras Introduces Its 2-Exaflop AI Supercomputer". IEEE Spectrum.
  65. Cherney, Max A. (2023-08-30). "UAE's G42 launches open source Arabic language AI model". Reuters. Retrieved 2023-10-08.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.