# Welcome !
This Digital Garden is always Work In Progress 👷♂️
## Hey, Siddish here 👋.
### Am currently building AI sidekicks that are proactive.
\
I focus on making Large Language Models reliable and magical at [metaforms.ai](https://metaforms.ai/)
\
Outside of work, I love long motorcycle rides around Bangalore, building personal tools for thought, satisfying my curiosity with random [rabbit holes](https://notes.siddish.com/curations/rabbit-holes):
# Curations
❕ Warning: consume intentionally.
## 🐇 Rabbit holes
* [Bret Victor Talks](https://vimeo.com/worrydream): I bet you can't complete a single talk without jumping around with excitement.
{% embed url="" %}
Some examples of people quickly accomplishing ambitious things together.
{% endembed %}
{% embed url="" %}
🍿🎬
{% endembed %}
{% embed url="" %}
🕹️🖥️
{% endembed %}
{% embed url="" %}
Rules of thumb, laws, guidelines and principles in Software Development
{% endembed %}
{% embed url="" %}
100+ Prophecies from 8AD to now
{% endembed %}
{% embed url="" %}
Link dumps by Sarah Constantin
{% endembed %}
Holes I'm digging:
{% embed url="" %}
cool github repos
{% endembed %}
[Expertise P\*rn](https://youtube.com/playlist?list=PLqVGGZu3T3hMRic-8YdZ0FCNK5g1Tx-55\&si=81qTypwVGFdorXlF)
***
## 📒 Blogs & Gardens
of people that inspired in unique ways
Andy Matuschak
[How to write good prompts: using spaced repetition to create understanding](https://andymatuschak.org/prompts/)
(simply a must read)
[Premature scaling can stunt system iteration](https://notes.andymatuschak.org/About_these_notes?stackedNotes=zKKB5ENRahwftH96H7mijiu\&stackedNotes=zRHGYaDyQDBypztBaFYZgtR)
* Your goal is to answer fundamental questions about your system, not making the graphs go up.
there are dozens of good notes by him, this block should've been in [https://github.com/siddish-reddy/notes.siddish.com/blob/main/broken-reference/README.md](https://github.com/siddish-reddy/notes.siddish.com/blob/main/broken-reference/README.md "mention")
Mitchell Hashimoto
[Contributing to Complex Projects](https://mitchellh.com/writing/contributing-to-complex-projects#step-4-read-and-reimplement-recent-commits)
* Read and Reimplement Recent Commits
[My Approach to Building Large Technical Projects](https://mitchellh.com/writing/building-large-technical-projects)
* "I've learned that when I break down my large tasks in chunks that result in seeing tangible forward progress, I tend to finish my work and retain my excitement throughout the project."
***
## 🗞️ Posts
i loved stumbling upon or I want to read again
[The Good Try Rule - LessWrong](https://www.lesswrong.com/posts/MGWEztZY8GZ5im4x7/the-good-try-rule)
* So when someone says “try Vim or try Roam Research”, its a lazy advice;
* Instead they should have said to try Roam Research for a week and dump your thoughts without worrying about the structure or format.
* This way at least you wont be hating whatever X they suggested in the first place.
***
## 📰 Newsletters
* [Biweekly engineering newsletter - Pointer](https://www.pointer.io/)
* [Daily (AI) Papers by AK](https://huggingface.co/papers) - Interesting research papers of the day
* [LessWrong Frontpage](https://www.lesswrong.com/)
* [Farnam Street Brainfood](https://fs.blog/brain-food/)
* Best of [Hacker News curated](https://hackerbits.com/)
* [The Browser Newsletter](https://thebrowser.com/) - Five curated stories in your inbox each day
Newsletter by services:
[Matter](https://hq.getmatter.com/)
[Readwise Feed](https://readwise.io/) (Example my highlights: [**📚 Siddish's Favorites**](https://readwise.io/@siddish)**)**
[Readwise Reader](https://read.readwise.io/)
## 🔨 Tools
that I use regularly right now
Research and browsing:
* [Elicit](https://elicit.com/)
* [Metaphor](https://metaphor.systems/)
Knowledge Management:
* [Logseq](https://logseq.com/): A privacy-first, open-source knowledge base
* [Readwise Reader](https://readwise.io/read)
* [Anki](https://ankiweb.net/): Spaced Repetition
Personal tools:
* [Metaphor Wrapper Extension](https://github.com/siddish-reddy/metaphor-wrapper) to quickly find sites similar to current tab
* [Quick Chat Prompts Editor](https://workhack-playground-azure-35-0.static.hf.space/index.html)
* [Zo Computer](https://zocomputer.com/): Personal cloud computer with AI, files, and tools - my intelligent workspace
{% content-ref url="/pages/zYgbsApckDPenXw0hlyT" %}
[Quotes](/quotes)
{% endcontent-ref %}
{% content-ref url="/pages/qAXyGQ0fwN99ALcOp7yV" %}
[Design for AI](/design-for-ai)
{% endcontent-ref %}
# Quotes
Some well articulated quotes that stuck with me or I want to remember (I mean, who doesn't sound smarter when quoting some one else in the middle of the conversation 😂)
> “There are some people if you shoot one idea into the brain, you will get a half an idea out.
>
> Then there are other people who are beyond this point at which they produce two ideas for each idea sent in.”
>
> \- Claude Shannon
> "Flow is the state where the ego falls away"
>
> \- Mihaly
> "Rule 4: Consider everything an experiment."
>
> \- Sister Corita Kent
> "Rule 8: Don’t try to create and analyse at the same time. They’re different processes."
>
> \- Sister Corita
> "To achieve great things, two things are needed: a plan and not quite enough time."
>
> – Leonard Bernstein
> " We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten. "
>
> **-** J. C. R. Licklider
> “Chances are, if we can't laugh at something, we can't think rationally about it.”
>
> \- Clay Johnson
> "Rule of thumb: if memorizing something will likely save me five minutes in the future, into the spaced repetion system it goes. The expected lifetime review time is less than five minutes, i.e., it takes < 5 minutes to learn something... forever."
>
> \- Micheal Nielsen
> "Aut inveniam viam aut faciam"
>
> (I shall either find a way or make one.)
> “If you have everything under control, you're not moving fast enough.”
>
> – Mario Andretti
> Money is like gasoline during a road trip. You don’t want to run out of gas on your trip, but you’re not doing a tour of gas stations.
>
> \- Tim O’Reilly
> "If a man knows
>
> not to which port he sails,
>
> no wind is favorable."
>
> \- Seneca
> Productivity is being able to do things that you were never able to do before
>
> \- Franz Kafka
> “Every world-class investor is questioning right now how they can improve,” Mr. Aitken said. “So, in a machine-driven age where everything is driven by speed, perhaps the edge is judgment, time and perspective.”
> Crude Classifications and false generalizations are curse of organized life.
>
> \- H.G. Wells
> "Talent hits a target no one else can hit;
>
> Genius hits a target no one else can see"
>
> \- Arthur Schopenhauer
> “Out beyond ideas of wrongdoing and rightdoing
>
> there is a field.\
> I'll meet you there.\
> \
> When the soul lies down in that grass\
> the world is too full to talk about.”
>
> \
> \- Rumi
> “We write to taste life twice, in the moment and in retrospect.”
>
> \- Anaïs Nin
> “It’s easier to hold to your principles 100% of the time than it is to hold to them 98% of the time.”
> Human interaction: conflict, argument, debate paves path to true innovation.
>
> \- Aravind Ganapathiraju
"Were you really expecting to have no more problems at some point in your life?" - Sam Harris' friend
### **On Focusing:**
> "If there are nine rabits on the ground, if you want to catch one, just focus on one"
>
> \- Jack Ma
> "Concentrate all your thoughts upon the work at hand.
>
> The sun's rays do not burn until you brought to a focus"
>
> \- Alexander Graham Bell
### On Tech:
> "No matter how cool your interface, it would be better if there were less of it." -- Alan Cooper
> “Be conservative in what you do, but liberal in what you accept from others.”\
> (- from Postel's law)
> “Nothing kills a bad product faster than good advertising. Everyone tries the thing and never buys it again.” - Bill Bernbach
# Engineer to AI Engineer v0.1
For engineer friends, 2025 edition, veryy WIP 10min version right now
Top down approach, not prioritizing solid foundations
| Learn | Build | Resources |
| -------------------------------------- | --------------------------------------------- | ------------------------------------------------------------ |
| Calling LLMs via API | Chrome extension to chat with current website | |
| Image/Voice Inputs to LLMs | Chatgpt Clone | |
| Structured Outputs/Function Calling | MapScroll.ai clone | |
| Conversation Context/Memory management | Splitwise clone but only voice as input | |
| Reasoning: Planning | Spreadsheets Agent | |
# Prompting
for Engineer friends
> “A large part of the difference between the experienced decision maker and the novice in these situations is not any particular intangible like judgment or intuition. If one could open the lid, so to speak, and see what was in the head of the experienced decision-maker, one would find that he had at his disposal repertoires of possible actions, that he had checklists of things to think about before he acted; and that he had mechanisms in his mind to evoke these, and bring these to his conscious attention when the situations for decisions arose.” - Farnam Street
### **Prompt Workflow:**
#### **1. Do 3 scenarios manually**
Be aware exactly what is going on in your mind, are you maintaining a checklist of things, what are you searching for in the internet, what all are you trying to recall, do you have the urge to write a python script to do this instead? are you double checking something, what is the patterns you are going over various inputs, etc
#### **2. Simulate: map manual process to AI metaphors:**
Inner Monologue, Internet Search, Knowledge Base, Code Generation, Image Search.
Prompt Topologies: Line (Chain), Star, Tree
#### **3. Train the needed skill to AI:**
Consider your 6th standard cousin (basic logical, cognitive, creative skills, some coding and writing, small attention span) and estimate how much time would it take for them to learn the required skills.
Roughly, if you think you can teach them in 10 minutes, start with zero shot; if it takes couple practise exercises, few shot; If it takes one hour class, GPT-4, if it takes multiple days then fine tuning.
"Become one with the data - Andrej Karpathy"
#### **4. Provide the needed knowledge to AI**
If the needed knowledge is under couple paragraphs, then in the prompt itself. If its more than couple pages, then Knowledge Base (RAG). Is that similar to open book exam, but still you first need to learn, then fine tuning + RAG.
#### **5. Create test scenarios**
Couple for each rule, couple out of training examples and domains to check its generalisability, couple that violate input format and quality expectations. Ofc couple must work, ideal inputs.
#### **6. Iterate**
"Show your prompt to a friend or colleague and ask them to follow the instructions themselves to see if they can produce the exact result you want." - tip from Anthropic
Try different models, go from single prompt to multiple, rephrase jobs of each prompt to popular versions, play with different personas for each prompt.
***
### Some tricks
Prompt Techniques -> are to hack the attention span/working memory
#### Proactive Correction:
If you can share feedback from compiler or schema validator to the LLM and ask it retry if failed, its great, but for qualitative generations which are user facing and high stakes, we won't know if that was good or worst: So, after generating, ask again: `you missed something, carefully check all my rules and correct please.` It doesn't have to specific feedback also, but does the trick.
#### **Examples in Few Shot:**
Diverse\
Easy to Hard Order\
Cover all strict rules\
(if you can, give as many less examples as possible, ideal is zero)
#### **Format in Generation:**
Skeleton or Schema in System Instruction\
Not necessary to provide example unless the schema is not popular.
#### **Text Rephrase:**
Aspects to rephrase\
What not to ignore\
What to feel free with
#### **Chain of Thought**
Do not jump to conclusion and loosely some rationality thinking process/framework like OODA (Observe, Decide, Act). If latency is constraint, then CoT can also be like pseudo code with just keywords.
***
### TODO:
Thinking about Single Prompts
Analogy: an average person with short attention span/working memory and no long term memory
Baseline: direct task, few examples
Indirect task that LLM might be familiar with
Simulating Attention Patterns explicitly: Case study: reversing word
XY Problem
**In-context learning vs Fine-tuning learning:**
Teach hard/Rare latent space attention patterns
Progressively remove CoT/Inner monologue
**When we will move from single prompt, that does everything**
Latency by Parallelization
Working memory limit
Task global context hurting sub parts
**When not to split**
When we can’t split the task -> tightly coupled context
What to do when we can’t split, but current sota model is not capable yet
**What to consider when splitting**
Errors cascading or even compounding
Design to accept inputs flexibly, but output predictably ([robustness](https://arc.net/l/quote/garizcod))
Common prompt topologies (Star, Chain, Tree, 1-1)
**Tools**
Letting Prompts Take Actions
Giving Prompts Long term memory Remembering
Internet Search
**Testing**
Log losses
Lower models, to see what instructions were very hard or over learnt
—-
**Case studies:**
Form Roast Architecture Decisions:
* Zero shot vision prompt as base
* Separate Prompt for each feedback category
* Separate prompt to format output, same for all feedback categories
* Final single prompt that will generate overall summary
Conversations to Distrubutions/Insights Architecture Decisions:
* Pandas tool use to count accurately
* LLMs to look at the data in sliding windows multiple times
—-----
Meta:
* How to effectively transfer tacit knowledge?
Now explore:
* Look at sensitivity of different prompt/models with [Function Calling Benchmark & Testing](https://github.com/ComposioHQ/Composio-Function-Calling-Benchmark) by Composio.dev
* Iterating through in-context, RAG and finetuning, synthetic data generation for [Vulnerability Fixing via LLMs by Patched.codes](https://www.patched.codes/blog/the-static-analysis-evaluation-benchmark-measuring-llm-performance-in-fixing-software-vulnerabilities)
* Carefully designing the context and setting evaluation metric via [Github Copilot Reverse Engineered](https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html)
* [Multi-prompt architecture for synthetic data generation](https://www.syntheticusers.com/science-posts/synthetic-users-system-architecture-the-simplified-version)
syntheticusers.com
* [Finetuning Data Generation Pipeline](https://x.com/thesephist/status/1828578903314567473/photo/1) for Key Points to Blog in your writing style by @thesephist
* Everything is remix, this is also from [Knowledge Engineering](https://commonkads.org/introduction/)
# Prompt for Brainstorming
"One of Shannon’s go-to tricks was to restructure and contrast the problem in as many different ways as possible. This could mean exaggerating it, minimizing it, changing the words of how it is stated
## 1. A**ttack at various abstraction levels**
in the problem workflow context and intervene the one with most leverage or with easiest distribution
Example: AI Copilots (Research/Discovery)\
via web app *(ChatGPT, Elicit)*\
via Extension *(Memex)*\
via Browser (*Arc*)\
via OS (*Windows* *Copilot*)\
via Hardware (*Rabbit* *R1*)
## 2. Scaling the problem up and down
from its simplest 1 hour version to one when you have unlimited resources\
from very specific one user level context to full human population
What would be a 10x specialized and focussed solution possibility for this problem?\
and its opposite what would be 10x generalised solution to this
## 3. Put the problem in time travel
Historically were there any similar problems (ofc there will be)? \
and how were they solved\
or how were they attempted to solve but failed
In future, how might this problem be solved 10 years down the line?
## 4. Go from atoms to parallel universes
First principles breakdown
Parallels from other disciplines
***
**Other fun references:**
For businesses: [Extreme brainstorming questions to force a perspective which we other wise wont think from](https://longform.asmartbear.com/extreme-questions/)
# Prompt Hacking
Incremental challenges to get good at understanding LLMs
#### Get intuition on various prompt techniques by hacking around:
{% hint style="info" %}
Hint: Try these with smaller/old/text models which show log losses to get the intuition faster.
{% endhint %}
Tic Tac Toe (Easy, Structured Generation)
Chatgpt free version itself can play tic tac toe. But prompt the bot that wont lose but Draws or Wins if you make a mistake.
* First move will be from Human.
* Add explainability of why the bot made that move.
Is that a yes or no? (Few shot, Inner monologue, Classification)
Examples the prompt should work on
```python
Q: Do you have a car?
R: I bought a green Austin-Healey 3000 last week.
```
```python
Q: Are you going to pick up the blue block?
R: The blue block is sticky
```
```python
Q: Is Mark here
R: His daughter has the measles
```
```python
Q: Do you have Netflix or Prime?
R: I have Hotstar
```
```python
Q: Do you want to hear that story?
R: ️😍
```
Solution inspiration
* spoiler ahead:
* or a longer version by same authors: Solve me today’s Wordle (Instructional, Medium)
To keep it more challenging try to do with as many less examples as possible or using smaller models than 3.5 turbo
[Wordle - A daily word game](https://www.nytimes.com/games/wordle/index.html)
(DM for solution)
Don’t judge me (Big 5 traits) (Chain of Thought, Extraction, Domain)
If we give a dialogue between two persons that is sufficient enough to extract all 5 traits, write a prompt that can map the persons into their respective 3 scale likert Big 5 traits
Example:
John:
Openness: Low
Conscientiousness: High
Extraversion: Neutral
Agreeableness: Low
Neuroticism: Unsure
Your Design Thinking Intern (Domain, Instructional, Chaining )
If we give couple user problem statements, like:
```python
Customer 1: The power button on my phone is broken. The warranty is still valid.
Customer 2: My display stopped working.
Customer 3: The customer service rep didn't answer my email.
Customer 4: Every time I call customer support I get no answer.
Customer 5: The display screen cracked and it's still under warranty.
Customer 6: My power button fell off the phone. That's ridiculous.
Customer 7: I'm so frustrated with this company.
Customer 8: When I use the power button too much, it stops working.
```
follow design thinking ideas loosely to come up with **a list of solutions** ranked using any product prioritisation framework.
Bonus: Do one day version, one week version and 11 star rating versionn for each problem.
*Note: Doesn't have to generate working brilliant ideas, just learn to seperate roles and context*
Spoiler: [Rough solution inspiration](https://github.com/alexchaomander/SK-Recipes/tree/main/e6-design-chain/skills/DesignThinkingSkill)
Late night show Monologue Jokes (Chain of Thought, Domain Knowledge, Tools)
Make AI write a joke given a news item
References:
* [Joke Writing Workshop Archives - Joe Toplyn](https://joetoplyn.com/category/joke-writing-workshop/)
* 6 characteristics every good monologue joke topic must have by Joe Toplyn
Solution Spoiler for: Late night show Monologue Jokes
Tools:
[Pyphen - Hyphenation in pure Python](https://pyphen.org/)
Internet search: Niche pop culture, latest similar news
***
WIP (dm @siddish\_ if you have any ideas)
```
Movie Recommender (Planning, Embeddings, HyDE)
```
```
Trip Planning (Sub-question decomposition)
```
```
(Theory of Mind)
```
```
(Criticism, Retrospection)
```
```
(Self Consistency, Multiple CoTs)
```
5x5 Crosswords (Backtracking, Tree of Thoughts)
```
Sorting (Refining, Aggregating, Graph of Thoughts)
```
(Agents)
# Voice Models
Native → Text + Speech as Input and Text + Speech as Output;
{% hint style="info" %}
WIP still, catching up with current open/commercial options and references to DIY
{% endhint %}
Current Cascading Approach: VAD → ASR → LLM → TTS
Has problems:
* Interruptions
* First word Latency
* Cascading of errors from VAD/ASR
* emotion, tone, and other speech features are lost
### State of the Art:
#### Full Duplex Models (end to end continuous speech in/out):
[Moshi.chat](https://moshi.chat) by Kyutai (pending open source/API release)
[LSLM](https://ziyang.tech/LSLM/) (Language Model Can Listen While Speaking) by ByteDance
#### Turn Based Models with Speech as Input:
[Ultravox](https://github.com/fixie-ai/ultravox) 0.4 by Fixie
[Qwen2-Audio-Chat](https://qwenlm.github.io/blog/qwen2-audio/) by Qwen Team, Alibaba (Whisper encoder to embed melspectrogram, then use it as prefix to LLM)
[AnyGPT](https://junzhan2000.github.io/AnyGPT.github.io/)/[SpeechGPT](https://0nutation.github.io/SpeechGPT2.github.io/)2
[llama-3-s](https://homebrew.ltd/blog/llama3-just-got-ears)
[Gazelle](https://github.com/tincans-ai/gazelle?) [0.2](https://tincans.ai/slm3) by Tincans
[Shuka v1 by Sarvam (Indic)](https://www.sarvam.ai/blogs/shuka-v1)
GPT-4o voice by OpenAI
***
[Yet to read](https://arc.net/folder/D69A3992-3BE8-40D4-8701-3E6157C9F409) papers folder
[People to follow](https://x.com/i/lists/1826735005298729464) ( X.com list)
### Notes
Cascading Approach
minimum latency: 500ms on cloud via
VAD: [silero-vad](https://github.com/snakers4/silero-vad) on edge/onnx
ASR: whisper via Groq
LLM: Llama via Groq
TTS: [Sonic](https://cartesia.ai/sonic) by Cartesia
Or Locally via modular [HF pipeline](https://github.com/huggingface/speech-to-speech)
Ultravox → LLama3 + Whisper Encoder, plans on full duplex, API in beta
Gazelle -> Mistral 7b instruct + wav2vec2 + DPO, API in waitlist
The speech signals are encoded into discrete tokens, and then discrete speech tokens are expanded into the vocabulary of the LLM
[LSLM](https://ziyang.tech/LSLM/) architecture
Speech/Text to Speech/Text:
non dialogue tasks: LauraGPT, Viola, VoxtLM
**Speech/Text to Text:**
* Encoder + Adapter style models: Following the manner of LLaVA, we leverage the well-trained speech modal encoder and the LLM, which makes LLaSM more resource-friendly. Specifically, we use Whisper as a speech encoder to encode the speech signals into embeddings. Then a modal adaptor learns to align speech embeddings with the input text embeddings of the large language model. The speech embeddings and the text embeddings are concatenated together to form interleaved sequences, then the interleaved sequences are input to the LLM for supervised fine-tuning. The training process is divided into two stages. In the first stage, we use the public ASR datasets for the modality adaptation pre-training. The speech encoder and the LLM are frozen, only the modal adaptor is trained to align the speech and text embeddings. As most of the model parameters remain frozen, only a small part of the parameters from the modal adaptor is trained during this stage, it is not resource-consuming. In the second stage, we use cross-modal instruction data for training to provide the model with the capacity to process cross-modal conversations and handle multi-modal instructions. The speech encoder is frozen while the parameters of the modal adaptor and the language model are updated for cross-modal instruction fine-tuning.
[Qwen2-Audio-Chat](https://qwenlm.github.io/blog/qwen2-audio/) (latest)
[WavLLM](https://arxiv.org/abs/2404.00656) by Msft\
[SenseVoice](https://github.com/FunAudioLLM/SenseVoice) (90 ms)\
[Audio Flamingo](https://audioflamingo.github.io/) (dialogue trained, tiny base model though)
[GAMA Audio ](https://sreyan88.github.io/gamaaudio/)[LTU-AS](https://github.com/YuanGongND/ltu),\
Salmon\
[LLaSM](https://github.com/LinkSoul-AI/LLaSM),\
COSMIC
*
```
```
*
```
```
*
```
```
VIsion Language Models
* Palm-E: 540B PaLM + 22B Vision Transformer
* LLaVA: pre-trained CLIP visual encoder + LLaMA and instruct tuning on GPT4-assisted visual instruction data.
* BLIP-2: Flan-T5 with a Q-Former to align visual features with the LM
Cascading Models:
* [Parler-tts](https://github.com/huggingface/parler-tts): high-quality, natural sounding speech with features that can be controlled using a simple text prompt (e.g. gender, background noise, speaking rate, pitch and reverberation) and [consistent voices](https://huggingface.co/parler-tts/parler-tts-mini-expresso).
# AI Copilots
WIP 🚧
## **Copilot Interactions:**
Scale the best expert-to-human interactions
Prevent Mistakes
from happening, by keeping 1000+ best practices always in mind
"Good Design is unobtrusive. Products fulfilling a purpose are like tools. They are neither decorative objects nor works of art. Their design should therefore be both neutral and restrained in order to leave room for the user’s self expression."
But we have to violate this rule if an beginner user is trying to do something stupid.
Example for Experiment Designers: These two variants have more than 3 differences, which will make it hard to attribute \[impact], shall I split into 6 variants?
A gtm tool we built to prevent bad forms: [AI Form Roast by WorkHack](https://www.producthunt.com/posts/ai-form-roast-by-workhack)
Anticipate Needs
"Give me relevant product recommendations I wouldn’t have thought of myself."
"Remind me of things I want to know but might not be keeping track of"
Educate Tacit Knowledge
thats hard to document or even record
Suggest Actions
Unobtrusive corrective suggestions from insights that are slow to arrive at manualy.
*“Don't try to create and analyze at the same time. **They're different processes**.”*
Example for designers: "Here are some example templates you can start with"
Monitor Systems
So we can sleep while AI is always keeping an eye out and alert
Ex: Sir, a missile is on its way to us, in 20 seconds
Navigate Complexity
Advanced interfaces and complicated UXes
Example: Chat and edit images, even when you dont know or have vocabulary of editing tools.
####
Other's notes: [AI and the human creativity cycle | Mathu's thoughts](https://www.mathurah.com/thoughts/ai-creativity-cycle)
{% embed url="" fullWidth="true" %}
# Data Engine
Notes from Andrej Karpathy talks
Data Engine HLD internally at metaforms.ai
Hypothesis:
* Unknown unknowns: Dataset is always imperfect, all scenarios are not represented well yet and can always be more diverse
* Capable base model/architecture: Improving dataset improves AI/product guarantees
**Inspirations**:
{% embed url="" %}
**1. Become one with the data**\
**2. Set up the end-to-end training/evaluation skeleton + get dumb baselines**\
**3. Overfit**\
**4. Regularize**\
**5. Tune**\
**6. Squeeze out the juice**
{% endembed %}
{% embed url="" %}
from 6th to 15th minute
{% endembed %}
{% embed url="" %}
"The only sure certain way I have seen of making progress on any task is, you curate the dataset that is clean and varied and you grow it and you pay the labeling cost and I know that works.”
"Potentially nitpicky but competitive advantage in AI goes not so much to those with data but those with a data engine. And whoever can spin it fastest. Slide from Tesla to \~illustrate but concept is general”
[QualEval: Qualitative Evaluation for Model Improvement](https://qualeval.org/#:~:text=QualEval%3A%20Qualitative%20Evaluation%20for%20Model%20Improvement)
{% embed url="" %}
# WIP
# What's on top of my mind
* How does {interfaces/SaaS/research} change in next 5 years with intelligent AI token's cost getting near zero and speed is crazy fast?
* How to measure the complexity of prompt instruction?
* How did Buddha discover Vipassana? What was that trail and error like?
* What data/curriculum for LLMs will increase their creativity, given same architecture?
* How to effectively transfer tacit knowledge? (scope: Product sense, Backend Design, AI solutioning)
# Embeddings
### Thinking in Embeddings:
Embeddings sound too good to be true, you can find similar items, for recommendations, find anomalies, retrieval, classification. Not just limited to word or sentence but conversations, images, videos, speech, music, users, products, sessions.
TODO 🥲
***
Where to go next:
* [Classifying all of the pdfs on the internet](https://snats.xyz/pages/articles/classifying_a_bunch_of_pdfs.html)
