I have been coding since 2022. I am comfortable with backend and frontend, databases, and setting up servers. LLMs are now everywhere and I know nothing about it. So I want to try building something with it. This article will be a way for me to document my learnings and progress along the way.
What the hell is an LLM?
Date: Nov 14, 2024
Today was the first day. I spend a long time talking with ChatGPT about what are LLMs and how to set them up on a server, or on my own computer.
I had heard that they come pre-trained but never really understood what that meant.
Then I learned that the best way to use LLMs is with Python, and not JavaScript. That's a good and a bad news. Bad news: I know less about Python. Good news: I am gonna learn. Digging a little more into that, I realized that I will need to build two backends. A node JS one that will act as my middle wear and connection to my DB. Then I'll have a python backend that will handle everything related to the LLM work. All this will communicate with APIs to my frontend, i'll use react for that.
After further research, I landed on Ollama and Hugging Face. Created an account on both tools and explored. They both look like powerful tools combined with a community. I don't fully understand yet what these tools really do.
Trying Ollama was really fun. I downloaded the lightest version of LLama 3 onto my computer. And using the command line interface, I was able to converse with the AI while my wifi was off. This was an incredible experience. I asked a ton of questions and the answers were very good.
This brought me to Youtube where I watched a couple tutorials on how to set up an LLM without Ollama on your own desktop and how to use a python env to communciate with it.
Something that is now on my mind is, what is the tool that I will be building. I need something easy to build but I am also looking for a coding challenge that will teach me something. Most importantly, I need to build something useful to a group of people.
Installed Llama 3.2 and got an answer
Date: Nov 19, 2024
This weekend, I set up Llama 3.2 on my machine to avoid relying on third-party apps like Ollama. After creating a Python virtual environment and testing basic scripts, I initially downloaded Llama 3.1 8B from the official website (8B only, because I want to be able to test things fast).
However, converting the model weights for Hugging Face compatibility was tedious.
This led me to learn that I could download Llama from Hugging Face directly and it would already be compatible with all the great tooling that HF has created. I discovered that Hugging Face already provides pre-configured models. HF "provides pre-configured models ready for immediate use with their transformers python library. This saves time, eliminates manual conversions, and ensures compatibility with their robust tools for tokenization, inference, and optimization."
So I deleted everything and started over to switch to the Hugging Face version, installed their CLI tool, and downloaded Llama 3.2. My test script worked perfectly, generating an answer to the prompt: "Explain the basics of UX design."
from transformers import AutoTokenizer, AutoModelForCausalLM
def load_model(model_path):
print("Loading model...")
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)
return tokenizer, model
def generate_response(prompt, tokenizer, model):
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs)
return tokenizer.decode(output[0], skip_special_tokens=True)
model_path = "my/own/path/to/my/llama_model"
tokenizer, model = load_model(model_path)
prompt = "Explain the basics of UX design."
print("Prompt:", prompt)
response = generate_response(prompt, tokenizer, model)
print("Response:", response)
Next steps: The model works, but response times are slow. I plan to optimize performance and build a simple front-end interface to streamline interaction. HF tools should make these improvements more manageable.
Creating the frontend with React
Date: Nov 20, 2024
This got to be my most beautiful design yet.
Yesterday and today, I am create a React frontend app so that I can start playing around with the AI in a easier way. The reason is, I am still new to the terminal and how to navigate commands, so testing Llama using the terminal is not really efficient for me. Another good reason is that I want to get better at front end and my final app with need a UI anyway.
There are so many more steps to "building a front end" that I originally thought. Installing React is easy, you just type npm install -g create-react-app. But that only installs the very basic things. I then added a router system so that I can go to different parts of my app by typing different URLs. Now, / is home and /about is a different React components.
After that, I started to build a list of different React components so that my code is more easily maintainable. For examples, the navigation, form, and the AI response, are all different "React Components", meaning, they all live in different files. After than, I installed SCSS, a version of CSS that makes CSS classes easier to manage. This part of the work was fun, very visual, so quite similar to what I am used to as a designer.
But then I needed to build a reliable state management system. That was a different story. Much more "tech" than what I am used to. I installed React Redux and created the necessary files for it to be set up properly: store, slices, etc. The purpose of having Redux on your react app is that you can more easily manage different data. Ex: I need to store the user question and then send that to the backend. And then get the backend answer, store that and then show it on the front end. All this data can be a mess to manage, that's what Redux solves. It sorts of manages and centralizes client information. (I think...)
Until next time.