My final week of the official DREU program! Overall this summer I learned a lot more about LLMs, medical simplification, annotations, factuality, and natural language processing research in general. Furthermore, I got to collaborate with very suportive, encouraging, and helpful peers and mentors. I will be moving back from Austin soon and really enjoyed my month researching in Texas!

Halfway through my time at Austin, I’m actually starting to get used to the blistering heat :)

I really enjoyed meeting Professor Li, grabbing coffee, and discussing the next project steps with Byron virtually in the office!

Moved to Austin! My first time getting scorched in 103+ degree weather throughout the day…

Moving to Austin, Texas soon!

I’ve tried FlanALPACA, GPT-3.5, ALPACA, FLAN-T5-XL, REDPAJAMA, FALCON, and 4-5 different prompts for each, as well as different parameter size variants for each model across the first ~200 abstracts in one of Byron’s trialstreamer datasets

What babysitting LLM generations in the middle of the night looks like:

Out of extreme frustration and hours spent on debugging conda environments, python compatability, un-installing and re-installing packages, I decided to purchase Google Colab Pro + I am so happy to not worry about the compute problems and that I can run many different LLM pipelines on different sessions at once :)

My second week of digging into LLM related research… I got the data of abstracts from Byron’s trialstreamer source, but am running into some extremely irritating compute limitations. To run these extremely large language models, of up to 10 billion parameters, I need to have access to High-RAM GPUs Currently I have access to two sources:

1. MIT Satori
2. Google Colab Free

Hi everyone! My first blog post! I’m really excited to start my DREU experience with Professor Li. I just had a first virtual group meeting and met Sebastian, a fellow student researcher at UT Austin. I also met Professor Byron Wallace.