How to Train a GPT From Scratch
An experiment reproducing NanoGPT and lessons learned
- Jan. 24, 2024 by
- Akash Kundu
We invite all users to submit feedback through the survey. The responses we receive will directly impact future development of Chameleon, including where to focus hardware procurement efforts and extension of testbed capabilities. The survey is a great opportunity to help guide the future of Chameleon! We will accept responses until Feb. 6, 2024.
Happy New Year! We are back and happy to see you are back as well! In this changelog edition we highlight the most significant testbed changes and upgrades from the last year and hint at some surprises yet to come. Looking forward to working with you in 2024!
Get the inside scoop on Chameleon Cloud’s common user issues and smart fixes in our latest 'Tips and Tricks' blog, highlighting a year of learning and problem-solving.
In this end-of-year blog, the Chameleon Team expresses gratitude for the community's support and achievements in 2023. Highlighting the innovative work showcased in user experiment blogs, the team invites more exploration. The blog also announces the holiday schedule for the Chameleon Help Desk and the exciting return of the "Chameleon for Christmas" double lease offer, allowing extended access to resources, including GPU nodes. The post concludes with warm holiday wishes and anticipation for continued collaboration in 2024.
Great news for the Chameleon Cloud Community! The IndySCC site, previously exclusive to IndySCC teams, is now accessible to all users until January 5th, thanks to the University of Colorado Boulder's support. With 164 nodes featuring powerful hardware, including 2x 12 core Haswell CPUs and 100G Intel Omni-Path, it's a fantastic chance to expand your research capabilities. Remember, new reservations won't be accepted after January 5th, so make the most of this opportunity while it lasts.
Delve into the intricate world of Multi-Level Erasure Coding (MLEC) and its application in large-scale data centers. Meng Wang, a Ph.D. candidate at the University of Chicago, presents comprehensive design considerations and analysis of MLEC, highlighting its advantages over traditional single-level erasure coding. The blog is aimed at exploring the significant impact of MLEC on data redundancy and storage efficiency.
We had a busy month at Super Computing earlier this month and with IndySCC. We also have been preparing for a virtual reproducibility hackathon next month. Changes this month include improvements to resource reservations, CHI-in-a-box, appliance fixes, and JupyterHub updates.
Chameleon Cloud's initiative for advancing scientific research through reproducibility invites the community to contribute to a growing repository of reproducible experiments. With state-of-the-art resources, Chameleon Cloud's Trovi service enables researchers to create and share their experiments as reusable artifacts. The upcoming Reproducibility Hackathon, a virtual event on December 15, 2023, offers an opportunity to transform experiments into valuable community resources. Users are encouraged to submit their Chameleon-based experiments for reproducibility packaging to enrich the platform, fostering a collaborative and innovative research environment.
This blog post outlines the experience of high school students engaging in a summer research program at NYU, focusing on cloud and edge Machine Learning inference projects utilizing the Chameleon platform and associated Trovi artifacts. The authors detail their practical exploration into machine learning at the cloud and the edge, review results, and discuss the technical challenges encountered and the solutions developed.