- The Hidden Layer
- Posts
- AI "Off-Roading": Week 1
AI "Off-Roading": Week 1
Week 19 of documenting my AI/ML learning journey (Feb 16 - Feb 22)
Brandon Kim
March 02, 2025
What was discussed last week…
I learned how to implement transformers in text generation.
For transformers to do this, I also learned how transformers can perform text vectorization to differentiate contexts and meanings of words/tokens!
I also delved into why topics in ML and DL talk so much about dimensionality and having so many dimensions.
I got sick of all of this learning, learning, learning. In all fields, application is everything; hands-on learning and stuff. “Experience is the best teacher”, as they say, and I need to start jumping into the ocean of artificial intelligence, no courses or anything (for now).
Just me and the internet, trying to figure it out.
Sunday, February 17th
So I decided to go for a text-generating transformer, since it was that I talked about last week, but…
:(
I had to properly install TensorFlow first, out of anything.
And so, after trying to install the correct one…
My computer sucks…I don’t have a CUDA compatible GPU (which is what regular TensorFlow uses)
What is going on…I decided to test my code anyway.
Sure enough, my computer still somehow doesn’t have TensorFlow?!
Oh wait…
Pylance Not Recognizing TensorFlow: If Pylance still indicates that TensorFlow is not found, ensure that your IDE (like VSCode) is using the correct Python interpreter where TensorFlow is installed. You can select the interpreter in VSCode by following these steps.
The yellow squiggly line under the “TensorFlow” went away!
So after using perplexity.ai as my coding crutch, I think I managed to get my libraries imported?
I did!
yay no error messages :)
Wednesday, February 19th
Ok, but now I can’t install or import other libraries, great.
Wait, but somehow…my computer runs fine even though with the yellow squiggly lines present (showing “unrecognized” imported modules).
I also looked up the TensorFlow Documentation about the get_file method/function to see what other texts I could import…and I settled on a dataset of iMDB reports compiled by Stanford:
Thursday, February 20th
After some more digging, I found another way to import text files into my file, using GOOGLE DRIVE?!
from google.colab import drive
drive.mount('/content/drive')
file_path = '/content/drive/My Drive/path/to/your/custom_file.txt'
with open(file_path, 'r') as file:
text = file.read()Sadly, this comes with a catch: although this works well with Google’s Colab environment, it doesn’t integrate as seamlessly into VSCode, which is what I’m using as my IDE.
Luckily, I did find another text file that can be imported using TensorFlow’s get_file() method, this time it’s about toxic comments, and detecting which ones are toxic:
url = 'https://storage.googleapis.com/jigsaw-unintended-bias-in-toxicity-classification/train.csv'
file_path = tf.keras.utils.get_file('jigsaw_toxic_comments.csv', url)Wait—it’s a CSV file… (meaning commas everywhere, oh no!)
At least it imported without errors…
…nevermind, I’m going to use a local file instead (via Python’s included read() method).
Also, new diagram drop:
It’s beautiful!!! (I’m probably gonna make it more stupidly detailed later)
Saturday, February 22nd
Only the file name of the .txt file was needed; the whole PATH of the .txt file didn’t work, because of something about being unable to read Unicode bytes or something…
The .txt file is about how transformers work, the info is taken from AWS
Wait, no I actually want to make it generate AP Stat FRQs (I got a 5 in 2024 when taking the AP Stat exam!) to make a tool for studying for the course!
But, that will be too complex for me right now…
Prompt: “How in the world do I train a transformer on AP stat FRQs?
Response: “Model Selection: Choose a pre-trained transformer model as a starting point. Given the specialized nature of AP Statistics FRQs, you'll likely need to fine-tune an existing model rather than training from scratch.”
Lessons Learned
I need to think of a project idea so I don’t gravitate just following another pre-planned project again.
Also, I learned that interpreters matter, whatever they are.
Resources
None…well maybe TensorFlow’s Documentation:
