How I use LLMs

Andrej Karpathy

How I use LLMs - YouTube

Tap to unmute

How I use LLMs Andrej Karpathy

Andrej Karpathy 1.38M subscribers

Introduction to Large Language Models (LLMs)

  • Overview of LLMs like ChatGPT and their development
    • Developed by OpenAI, deployed in 2022
    • Ecosystem expanded significantly by 2025
    • Examples: Gemini from Google, Co-Pilot from Microsoft, Claude from Anthropic, Grok from xAI
  • Chachi PT (ChatGPT) as the incumbent, feature-rich model
    • Other models have unique features not found in ChatGPT

Technical Fundamentals

  • Interaction with LLMs through text
    • Inputs and outputs as token sequences
    • Use of tools like Tokenizer to understand token sequences
  • Pre-training and post-training phases
    • Pre-training: Compressing internet knowledge into neural network parameters
    • Post-training: Fine-tuning to adopt assistant-like persona
  • Importance of understanding model limitations and knowledge cut-off

Practical Use Cases

  • Knowledge-based interactions
    • Example: Caffeine content in Americano (63mg)
    • Use of LLMs for medical queries (e.g., cold medications)
  • Importance of starting new chat sessions to maintain relevant context
  • Model selection based on task requirements
    • GPT-40, GPT-40 mini, and other tiers with varying capabilities

Advanced Model Features

  • Thinking models with reinforcement learning for complex tasks
    • Useful for math, code, and problem-solving
    • Examples of model performance in programming tasks
  • Internet search integration for real-time information retrieval
    • Use of tools like Perplexity and Chatbot Arena
  • Tool usage in LLMs
    • Integration of Python interpreters for advanced data analysis and plotting

Multimodal Interactions

  • Voice and audio interactions
    • Using system-wide apps for voice-to-text on desktop
    • Text-to-speech features in apps like ChatGPT
  • True audio processing within LLMs
    • Advanced voice mode for direct audio token handling
  • Image and video processing
    • Uploading images for analysis and generating images with DALL-E 3
    • Video capabilities in mobile apps, like identifying objects or text in video streams

Quality of Life Features

  • Memory feature in ChatGPT for personalized interactions
    • Storing user preferences and conversation history
  • Custom instructions for tailored model responses
  • Custom GPTs for specific tasks, like language translation or vocabulary extraction

Conclusion

  • Rapidly evolving ecosystem of LLMs with diverse capabilities
  • Importance of selecting appropriate tools and models based on tasks
  • Future developments in LLM capabilities and applications