Fable 5 Ban, Meta's AI Gulag, Elias Thorne & What is Loop Engineering?

Shimin (00:00) Hello and welcome back to Artificial Developer Intelligence, a weekly conversation show and a study session about AI and software development. We go through hundreds of links and dozens of newsletters each week, so you don’t have to. My name is Shimin Zhang, and today with me are my co-hosts, and Dan, popping up on Amazon as an author of alt medicine cancer handbooks and psychological thriller novella, Lasky.

And Rahul, he has significantly higher intelligence than third party contractors. Yadav. Hello friend, how are we doing this this lovely Tuesday? Did you guys watch the awesome UFC pay per view o over the weekend?

Dan (00:39) No, nor have I been able to watch World Cup because I don’t want to pay eighty dollars to watch it.

Rahul Rahul (00:44) AHHHH

Shimin (00:45) Fair enough. Yeah, me neither.

Rahul Rahul (00:46) It’s eighty bucks to

stream it.

It’s eighty bucks to stream it.

Dan (00:49) I don’t know. If you do YouTube TV it is and ⁓ yeah.

Rahul Rahul (00:53) See.

The the way I learned about the score of that is our neighbor was shouting so loud every time US was scoring that with windows closed, my wife and I were just like, Yeah, I think we scored another goal just now. The the whole the whole apartment building of hundreds of units knew when US was scoring.

Dan (01:00) That you just knew the score based on

Whoa.

Shimin (01:08) Ha ha ha.

Dan (01:14) Yeah.

Shimin (01:14) Amazing and nobody was working. Alright, on ⁓ this week’s show, we’re gonna start as always with the news threadmill where we’re gonna talk about the latest in the anthropic Fable Five drama and a continuation of Meta’s ⁓ AI workflow drama.

Rahul Rahul (01:17) Yeah.

Dan (01:35) Then we’re gonna move on to post-processing, where we’re going to talk about a mysterious character that appears on the internet, and we’ll see why. we’re also gonna be talking about how AI is moving quickly, but the economy is moving slower. And then finally, something a little more practical with loop engineering.

Shimin (01:53) And then Dan is gonna show us his ⁓ DS four, dwarfsar four usage.

Dan (01:59) Maybe. Yeah. and then next up we’re gonna do a deep dive on a pretty interesting paper Called Beyond the Steeper Curve, AI mediated metacognitive decoupling and the limits of the Dunning Krueger metaphor. Say that two hundred and seventy eight times quickly.

Shimin (02:01) Maybe.

And as always, we’re gonna wrap the show up with two minutes to midnight where we’re gonna take a look at a couple of news items related to the state of the AI bubble. Well, let us get started. First up, Fable five, the latest, hottest model that nobody gets to use as of last Friday.

Dan (02:36) Well, I think last time we talked it had just come out, right? And we were we we were like I mean it was like literally hours I think before we recorded and then I hadn’t even played with it yet. I I got to use it for what exactly three days, I guess. Something like that. And then yeah, all of a sudden it went away at around what was it, five five twenty one PM Eastern time.

Shimin (02:39) Yes.

Dan (02:58) because the US government shut it down. And they cited a national security directive on well really it’s an export control, suspending all access to Fable V and Mythos V by any foreign national, whether they’re inside or outside the United States, including foreign national anthropic employees.

And so since obviously that’s a little bit tricky to enforce when you don’t have a system that handles that at all, they just shut the whole thing down.

⁓ and then planted messages all over their various clients. So you could see that Fable was unavailable and directing you to the official statement about it.

Shimin (03:36) Right. So I guess before we dive into the end of the Fable access, like Dan, what did you think of Fable in the couple of days that you actually get to use it?

Dan (03:44) I didn’t so I mean it’s mostly vibes, right? Because we’re not running like evals on it. ⁓ like vibes seemed good, but not like groundbreaking to me. But I didn’t really point at anything too hard, you know? It was just normal everyday little like snippety tasks.

So for that I didn’t see like a tremendous difference from four eight, but that’s again hu massive grain of salt. Didn’t do anything useful, didn’t challenge it at all. So who who really knows, you know.

Shimin (04:12) I had a close encounter with ⁓ I think AI psychosis with Fable Five. I was working with Fable F well it’s a it’s it’s the closest I think I’ve I’ve gotten with an AI model. if that is of any symptoms of ⁓ how powerful the model is. I was working with Fable 5 to design a novel drone technology.

Dan (04:18) It drove you to it?

Okay.

Shimin (04:35) So like a drone that flies in a different pattern than the our usual quadcopter approach. And I’m not an aerospace engineer, but ⁓ it seems to do a really good job. And at some point I was like, now I actually have to get a degree in aerospace engineering to actually know if it was hallucinating or talking, kind of just making things up. And that has not been my experience with previous models.

So and sampling size of one, I I find it to be a pretty major step up.

Rahul Rahul (05:05) It was good at exploiting human flaws in addition to software flaws, I guess. That’s what we learned. Yeah. It broke shimmer.

Dan (05:09) That’s why it’s so good at security.

Shimin (05:12) Yeah. It

it got through my defense, if nothing else. Yeah.

Dan (05:15) Yeah.

Shimin (05:16) Yeah, I also used it to help me do some research synthesis and as like a research agent and it it did a really good job too. So I am a believer. So

Dan (05:27) So are you gonna

pay API prices for it? Assuming they’re still gonna do that when

Rahul Rahul (05:33) If he

can design that quadcopter tech, it’ll be worth it.

Shimin (05:34) Yes.

Dan (05:37) It’ll pay for itself, obviously.

Shimin (05:37) Not for a qu not

for the quadcopter, of course, but ⁓ I would I would use it for ⁓ a similar kind of research synthesis and report writing data. I f I feel like that is the one use case it is worth the API price. Not not when it comes to day to day coding No, that it’s too good for that.

Rahul Rahul (05:40) Yeah.

Dan (05:56) I wrote a document for my boss this week and ⁓ sent it to him and then underneath it next line was I wrote this by hand ‘cause I did. I was like, I just feel like I need to state that these days when I do it, you know.

Shimin (06:10) Alright, so Fable 5 is gone for now. We’re gonna tell our imaginary grandchildren one day about that time we had access to Fable Five 5 turns out it’s a cautionary tale about ⁓ something or other all along.

Dan (06:25) Well,

do we want to get into the drama a little bit too? I don’t know. I don’t know how much we want to cover that. But yeah. So there’s been a couple of other developments since it was shut down. So Axios reported that Anthropic has flown a bunch of staff to DC to try to like diffuse the situation. And then there was additionally a I believe it was a security researcher that claimed

They got the inside track from someone that was in the reporting chain on this because supposedly AWS actually or a a ⁓ high-ranking person within AWS was the one person responsible for this because a security team at AWS discovered that a jailbreak was possible, supposedly. And then

⁓ they ratcheted up to I guess, I don’t know, their boss or something, who then reached out to the administration and was like, my gosh, you gotta shut this down. but someone that was in that chain or like had access to it claims that essentially all they asked it to do was quote unquote fix this code was the prompt that scared everybody. So it’s kind of like, how is that a jailbreak? You know? So that’s been sort of the latest development on it.

That was as of like today or yesterday that that piece was was breaking still. So as of recording time it’s still still banned, but ⁓ we’ll see how it goes.

Shimin (07:42) Yeah. Anthropic is negotiating with ⁓ the White House to try and get it unbanned, I guess. Yeah, this seems like the end of the universal access era of ⁓ the frontier models.

Dan (07:45) Yeah. Yeah.

Yeah, and that’s really thing that’s come out of this, right? Is is a lot of people are like I I’ve heard discussions, some people were talking about like, ⁓ I wanna run like really interested in running my own models now because like if, you know, this can y get yanked out from under me at any time, like that’s not okay. And then particularly folks outside the US have felt like they can’t like necessarily trust American inference providers after something like this because you never know.

you know, if it can be influenced by like effectively like a a government or maybe even a political decision. We don’t really know, but like maybe it was, then that doesn’t it’s not something you want to hinge your business on for sure. So

Shimin (08:32) Yeah, absolutely.

Rahul Rahul (08:33) Yeah.

And and the we already we’re already seeing in the news how, you know, running open source models is cheaper and everybody’s now in the cost cutting era. And so those already

the flow is going against models that are super expensive and you might be paying more for more than what you the value you’re getting from these models and on top of that if you add this as another reason why you don’t need to use these models at some point you’ll have enough in the list of why we must not use or we shouldn’t use the models from the labs and then people would default to open source models and deploying their own

versus using what the labs are offering.

And one thing I didn’t know was that they only got ninety minutes to comply, which I thought was crazy fast. Ninety minutes to cut the thing. Yeah. So

Dan (09:21) Wait, do I got what?

Really? Wow. That’s actually pretty impressive that they’re able to do it that

Shimin (09:28) Yeah.

Dan (09:31) fast. Cause like consider how much like how many moving pieces are rolled out, but

Rahul Rahul (09:36) Yep.

Shimin (09:36) ⁓ vibe coding works, is is the moral of the story.

Dan (09:39) I’m sure

Rahul Rahul (09:39) Yeah. ⁓

Dan (09:40) they

have some sort of r you know, I can’t be a company that big and be doing influence and not have like a pretty elaborate routing setup which probably has all kinds of like flag based control, you know. Yeah.

Shimin (09:50) Yeah, feature flags for sure. All

Rahul Rahul (09:54) It was interesting it was restrict restricted to foreign nationals because it makes the assumption that everybody who’s a US national is going to be smart and not mean any harm to the US. And it’s like watch any movie with enough money you can ⁓ you know, do whatever. So it it was interesting the the line that they drew on who shouldn’t get access.

Dan (10:01) Yeah, it’s like totally fine. Not not up to Yeah.

True.

Shimin (10:17) Yeah, that’s true.

Rahul Rahul (10:18) And

then Anthropic power i Anthropic did the better thing of just cutting access versus doing it that way. Yeah. Yeah.

Dan (10:23) It was really the only choice they had, you know, for better or for worse. But

Shimin (10:27) Yeah, given

ninety minutes for sure. Okay, now moving on. let’s take a look at an update from the meta drama brought to us by Rahul

Rahul Rahul (10:35) So Meta has this applied AI unit that is run by Alexander Rang, who used to run Scale AI and then moved to Meta last year, ⁓ sometime late last year if I remember correctly. and they’ve been through a number of layoffs. ⁓ that shouldn’t be news to anybody who’s been following the podcast. ⁓ one of the things that has happened as part of those layoffs is

⁓ people have gotten quote drafted end quote into this applied ai unit ⁓ where their job is to come up with problems that their models can solve because their models are not as good as let’s say cloud code ⁓ and and other ⁓ you know gpt models and whatnot to at solving these problems their rationale on using the

Shimin (11:07) Mm-hmm.

Rahul Rahul (11:24) w ⁓ on having meta engineers do this versus ⁓ outsourcing this work to contractor, which is what ⁓ Scaly I did, is that meta engineers are smarter than contractors. But what they’re doing essentially is having engineers write puzzles and coding problems and label basic data, ⁓ w while, you know, being paid meta salaries. And so

Shimin (11:47) Mm-hmm.

Rahul Rahul (11:48) People have gone to the extremes of calling it a gulag, which at the end of the day, you know, you should pick your words and read about the gulag before you pick. If this is gulag, you’re really not gonna ⁓ yeah, gulag r had free food and dentist on site and all sorts of things. So, but you know, their choice of words. ⁓ someone blew up recently in an

Shimin (11:56) hehehehe

Dan (11:58) A glug that has some pretty nice stock options. RSUs.

Shimin (12:01) Yeah.

Rahul Rahul (12:12) ⁓ internal call where they’re calling some senior executive, ⁓ some expletives and ⁓ it there’s been drama internally. So that’s what’s happening. P engineers are doing work that is not necessarily hard engineering work and it’s an interesting approach to training their AI.

Shimin (12:31) Yeah, I I got two thoughts on this. One, ⁓ which is the memo from Zuckerberg that states Meta’s North Star is to be the best place for the most talented people in the world to make an impact. if you disregard the ⁓ kind of impact you’re you’re causing, I guess that is still true. and B

Rahul Rahul (12:50) Well swimming.

it reminded me of the quote that ⁓ I’m trouble I’m having trouble hearing your words over the loudness of your actions. So

Yeah. That’s all we need to say on that.

Shimin (13:02) Yeah. And the second one is you know, people who got drafted into ⁓ the quote unquote gulag, ⁓ they’re getting a lot of experience, getting a lot of EXPs leveling up on AI model evaluation since they’re creating puzzles that the current generation of meta models cannot solve. So they’re getting some like potentially valuable hands on experience going forward. I don’t know. I without the details of those puzzles I’m not sure.

Yeah, how valuable that will be. But there’s potentially a silver lining all of this for them.

Dan (13:29) I suppose. I mean, if I if I were if I received an email out of the blue that said, welcome to the applied AI group, you’re being transferred on Xdate, I would have assumed just from the title that I was probably going to be working on like harnesses or something, you know? And then to find out that I was like writing puzzles and like basically leet code problems for a model, I might be a little bit disappointed as well.

Shimin (13:43) Mm-hmm. Yeah.

Rahul Rahul (13:44) Yeah.

Dan (13:54) But the other real question that this sort of tank tech crunch article that we’ve got all this stuff from didn’t go into is w are your mouse clicks and key keystrokes being logged as you create these puzzles?

Shimin (14:04) Yeah.

A hundred percent. A hundred percent.

Mm. Yeah, well we’ll see if the ⁓ workers end up, you know, doing something about these conditions, or is the sound of the RSU’s vesting just too loud that ⁓ I couldn’t quite hear them organizing. All right.

Dan (14:18) haha

Rahul Rahul (14:20) I I see

memoirs in the future about how bad things were at Meta, but only after you made a few million dollars from being in Meta. And ⁓ then like, guys, the gulag was so bad, I only made a few million dollars. I would like to milk it with some best selling business book now.

Shimin (14:27) Mm.

Shots fired.

Alright, well before we Yeah, before we ⁓ cut off all our hireability in the tech industry, let’s move on to post processing where Dan you have our first post this week titled ⁓ about Eliza Th Thorne? Eliza Eli Elias Elias Thorn.

Dan (14:39) Careful, you’re gonna get our podcast season desist.

Rahul Rahul (14:42) Yeah.

Dan (14:46) Yeah.

Elias, Elias Thorne. Yes.

So I haven’t personally noticed this, but ⁓ this article came up on four o’ four media, you know, sort of casually read, and I it was too interesting for me not to get hooked by it. They got me with their headline. So it’s chatbots keep telling stories about lout housekeeper Elias Thorne. We might know why. So apparently if you ask

pretty much any model. so everything, chat you know, the Chat GPT, Gemini, even Grok amazingly enough, to tell you a story. there’s something like I think it was an 88% chance, is that what they had said, that you will wind up with a story featuring one of three different characters, and Elias Thorne is one of them.

So they did kind of an interesting project which they roundup writing a paper about, ⁓ where I guess it was let’s see, Sill Sil Hamilton and David Mima at Cornell University’s Department of Information Scientists, science. they sampled two hundred or twenty thousand total stories from Chat GPT, Claude, and Gemini, using only five different prompts.

And they found that eleven words, which included Elias Mara and a couple others, kept coming up over and over and over again. And the part that really got me about this is that like, so I mean, I guess it’s no secret that like there’s a ton of like slop written stuff on Amazon, right? Be it ebooks or even in some cases like printed books. But there are entire books and in some cases Elias is considered the author.

of of these works that are like all over Amazon, which is pretty wild. But the part that was really interesting is like why did this happen? so I guess the the reasoning that they’ve come up with I don’t know if this was from the paper or if it was something that 404 dug of, but there was actually a data set that was published and it was based on

A million actual conversations with Chat GPT 3.5. And in those conversations, there were 111 chats, I guess, that had that featured this character where someone had like prompted it to write ⁓ stories about like a lighthouse keeper named this person or whatever.

So that data set became ⁓ like that those conversations were used in this like wild chat data set. And then in turn, that data set has been used for future generations of models across all of the model providers. So even though this sort of like originated from Chat GPT, it’s like sp almost virally spread because of the training set. so now Elias the the lighthouse keeper is is amongst us, whether we like it or not.

Shimin (17:25) Mm-hmm.

Dan (17:34) It’s it’s like a data set virus, they said, which is kinda cool. So yeah.

Shimin (17:39) Yeah, it’s kinda like a butterfly

effect. Yeah. unintended consequences of certain data sets being amplified over time.

Dan (17:43) Yeah, I mean first

first we had goblins and whatever and now we’ve got Elias. So it’s pretty interesting. But that does also kind of the other reason why I think it’s worth talking about is that it brings up one of the core, I guess, sc scary points about, you know, there only being three frontier labs and blah blah blah is like, well, if something this innocent can happen that effectively influences the output of all these

Shimin (18:02) Mm-hmm.

Dan (18:12) you know, Frontier Labs models, w what about like a actual concerted attempt to like poison the training data for a cer specific purpose, you know? could already be happening and we don’t know if it’s like a really sneaky actor. So just something to think about on your, you know, Thursday afternoon.

Shimin (18:25) Yeah.

Rahul Rahul (18:28) The

th there are for forums out there that have their own version of reality and if they’re on the internet, you how do you choose to, you know, train on them?

Shimin (18:38) Right.

Rahul Rahul (18:39) I I did a search while we were talking on Amazon for Elias Thorne. first one is The Legacy of the Pharaohs, the Elias Thorne Chronicles, Book Two. The very second recommendation 100 plus AI side hustles to make money and achieve financial freedom by Elias Thorne. Yes.

Shimin (18:54) Mm-hmm.

Dan (18:56) Is it was it was it was it written by Elias Thorne or what was the connection there? excellent.

Shimin (18:57) ⁓ we’re not the only ones with that idea, yeah.

Rahul Rahul (19:04) Elias Thorne is writing fiction, side hustles, ⁓ the cancer handbook you you had mentioned earlier, YouTube unlocked, how you can ⁓ crack the algorithm. Elias Thorne’s got a lot of knowledge, man. He very versatile.

Shimin (19:04) Amazing.

Dan (19:19) should hire him as the media manager for our our podcast.

Rahul Rahul (19:22) Ha ha ha.

Shimin (19:22) I do wanna see the a generalized version of of this study, right? Like look at word frequency of words on the internet from twenty twenty two, twenty twenty three till today and see what words. Why

Dan (19:34) But why would that matter?

Why does this

matter? Sorry, it was a bad joke. Why this matters?

Shimin (19:42) Why does it matter? Well well, why it matters, ⁓

it’s not just a frequent exercise in word frequency. ⁓ it is also a way to see how LLM has reflected upon our real world. Right? Like if I these days I actually get triggered whenever I see the word substrate. ⁓ either in like a conversation or reading it on an article, it’s like, God damn it. Yeah. No one uses substrate.

Dan (19:49) Yeah.

Mm-hmm.

Rahul Rahul (20:02) Yes.

I yeah, no one used it before until a year ago.

And now everybody’s, you know, using twenty dollar words.

Shimin (20:11) Yeah.

Right. Or the other one that triggers me also is hedging or hedge. ⁓ I don’t feel like I use that word I seen that word five years ago. Now it’s like area. Just say unsure. You don’t have to

Rahul Rahul (20:18) yeah.

Dan (20:24) Or

at least not without the word fund immediately after it.

Rahul Rahul (20:28) Yeah.

Shimin (20:28) Or or that,

yes.

Dan (20:29) But now we’re hedging everything. Why this matters? ⁓ well

Rahul Rahul (20:31) Why this

matters makes me think of ⁓ shout out Jack Clark. His newsletter has the cause the summaries always have the here’s the piece of news why this matters section every time and it I wonder if that’s how ⁓ you know they really pick that one up.

Dan (20:48) I mean the sad part is this probably reveals a lot about like the human well, I don’t even know. Is RL even human graded? I guess it is. No, okay. Yeah. At some point humans probably graded some part of this, right? And that caused some of this. I don’t know.

Shimin (20:49) Okay.

No. It depends, yeah.

Yeah. Yeah.

Alright, on to our ⁓ next post. speaking of AI generated writing, Rahul.

Rahul Rahul (21:12) ⁓ I actually do not know who this is by and I’m not sure if it’s attributed anywhere. So I will just call out the

okay. It it’s at the bottom actually. Built by Emory Taziki. the website is how fast is dot AI. And as the domain says, the website is all about how fast is AI actually. And ⁓ it’s about the speed of development in AI is very fast, but compared to how the economy moves,

you know, w what should we make of that? Because the economy only moves at a certain pace. We’ve seen that from the data from the last hundred and fifty years. We got railroads, we got electricity, we got all sorts of we got air travel, all sorts of these things. but the economy has moved at a rate of about two percent a year, ⁓ with all these transformational things happening over the past hundred and fifty years.

AI is developing very fast and the task completion horizons are doubling almost every six months. but would that actually mean that the economy moves any faster than the two percent? So that’s kind of the crux of what this is trying to get at. a lot of the there’s some other references on this website, but the majority of the research that this is based off of is coming from

Charles Jones or Chad Jones, I think he’s ⁓ based out of he’s a professor based out of Stanford. ⁓ and one of the things that he talks about, he also has a YouTube on this, ⁓ he talks about is this weak link theory, which is at the end of the day, the economy is moving at the pace of the weak links. and even if AI gets as fast as we want it to, ⁓ you know, as we all know, the ⁓ a chain is only going to be as strong as its weakest link.

⁓ w one of the like best business ways of looking at this is the whole fury of contra constraints by Eliahoo Goldwright where the bottleneck determines how fast ⁓ things are going to move, things are going to happen. And so if the human judgment is the weak link, if our decision making is the weak link or the accountability that we take on because you can’t really bring a system to court.

if those are the weak links, until they’re resolved or you s find a way to get them out of the loop, ⁓ the economy might still only grow at a steady pace because ⁓ those bottlenecks would still remain. one interesting thing that I learned from this I that that I didn’t know about is ⁓ we’ve had this, you know,

The explosion of microchips and computers ⁓ over the past couple of decades has been insane. But actually the US GDP has fallen from ⁓ the peak was four point three percent in two thousand to about three percent in twenty twenty-three. ⁓ because the prices have fallen, the share of GDP has actually fallen, which I never I thought, you know, computers are everywhere. You can like look anywhere within your living room even and you can point to almost anything these days.

And it’ll have a microchip in it. And so intuitively, ⁓ I thought the share of computers would be larger in the G D P but it’s actually ⁓ smaller. And then the scarce inputs obviously are still there. trust amongst humans, our coordination costs, our judgment and all of those things. they plan like

Shimin (24:30) Yeah, that chart really

surprised me as well. ⁓ like I think the E AI equivalent would be, you know, the demand for AI compute does not actually meet this trillions of dollars that we’re throwing into the infrastructure and CapEx. So per token cost might actually fall over the next fifteen, twenty years, right?

Rahul Rahul (24:50) Yeah, yeah. And then maybe intelligence is everywhere but the GDP at the at the end of the day. ⁓ yeah. And they do call it out, ⁓ i i in in this ⁓ website that, you know, sometimes some of the things that we can we feel but we c don’t really see in the G D P are things like we’re living longer now, we have vaccinations, right? So

Dan (24:57) Ha ha

Rahul Rahul (25:14) quality of life is better, all these things, does that necessarily reflect in the GDP? Not really, because it’s still like chugging along at two percent or so. So AI could be one of those things where our experiences change and a lot of other things change, but it doesn’t actually end up reflecting in the GDP. ⁓ which would be very interesting considering the the amount of money we’re spending on it.

Dan (25:35) There.

Well, that reminds me of the the there was a

post that I think I submitted a long time ago, like probably I don’t know, twelve weeks ago or something, that was like AI could be amazing, but also not a game changer in the same, you know, in the same breath, which is an interesting statement because you’re you look at it and you’re like, my god, how could it not change everything? But then you look at stuff like actual productivity numbers and

Rahul Rahul (25:50) Yep.

Dan (26:02) I don’t know, like the one thing we’ve seen the big the big uptick in is show hacker news posts, right?

Shimin (26:07) Yeah.

Rahul Rahul (26:07) True.

Shimin (26:09) bot sitting has been a trend this week on the internet, on the AI corners of the internet, where folks are spending more time yet sitting their bots ⁓ which, you know, doesn’t doesn’t actually necessarily result in higher productivity, right? You’re just changing the nature of the job. ⁓ so I find that to be interesting.

Dan (26:15) Not sitting, yeah.

Mm-hmm.

Rahul Rahul (26:28) What is bad sitting?

Dan (26:29) You’re just yes, yes, yes, yes, yes on your clawed prompts, you know, every

Rahul Rahul (26:32) ⁓ well they need to hear about

loop engineering a few minutes later in the podcast.

Shimin (26:35) Yeah. I will get into that.

And for software developers, I think it’s helpful for the listeners to take a look along with us, what are the weak links that ⁓ currently cannot be solved with ⁓ AI just yet, right? Like we have

Still system design and architecture, ambiguous requirements, integrating with messy production data, owning the outcome when it breaks. I feel like all of the four I could see system design and architecture being solved by tech. I could see integration with messy production data be solved by AI, but not so much the ambiguous requirements and owning the outcomes when it breaks. And those are things that both take a lot of time to actually sort through.

Dan (27:19) Mm-hmm.

Shimin (27:20) And also has a lot of responsibilities that are hard to just or perhaps, you know, impossible for now to ⁓ just push off to the AI.

Rahul Rahul (27:28) Yeah, and this also made me think of we had talked about Lewis Garricano, I think their name is one of the concepts they were talking about is how humans having residual decision rights. ⁓ at the end of the day a lot of what we do is we say, if this goes wrong, I’ll take ownership of it, right? So regardless of who did the work, as long as you’re the person who owns the consequences.

Shimin (27:40) Mm-hmm.

Rahul Rahul (27:53) How do you take that away? I I just honestly have no idea. And when the when it goes wrong, what do you do?

Shimin (27:56) Yeah. Yeah.

Dan (27:58) Yeah, and then the difference that we’ve talked about

before, like with running software, is then all of a sudden you potentially have this source of cognitive debt, right? Which is the L LM spitting out code that you haven’t read. And yet you still have the responsibility for it in both maintenance and in, you know, running the thing in production, whatever it is. So what happens then?

Rahul Rahul (28:06) Yeah.

Yeah. There could be a world where everybody just agrees to not have some of these things in there, but I don’t think then you’d be able to do anything critical.

Shimin (28:25) Mm-hmm.

I could see a world where the responsibility piece gets solved via insurance. But as we’ve seen in self driving cars, like it’s been around for t twenty years now and we still haven’t figured out like who exactly is responsible when something goes wrong. So ⁓ it probably takes us another twenty years to solve it for coding as well.

Rahul Rahul (28:34) Mm-hmm.

Yep.

yeah, I agree with that. And there’s we already s saw this in the world before AI and I will not name any names, but companies do do the math of is it cheaper to be sued for this and pay the lawsuit fine and still go ahead with this you know, whatever they’re doing, versus actually doing the work to actually comply with something.

And you can assume that calculation will keep happening where companies would rather you know, pay the fines, be non compliant and take the benefit of things moving faster. and as long as the the numbers work out, ⁓ I can see that role coming to fruition.

Shimin (29:22) Mm-hmm.

Yeah, basically Fight Club. ⁓ okay. Moving on. this week my post, speaking of foreshadowing, ⁓ is one by ⁓ Addy Osmani on loop engineering. let’s first kind of define what loop engineering is. So we all are familiar with ⁓ the Ralph loop.

Rahul Rahul (29:35) No.

Shimin (29:52) Where you essentially do a slash loop and have the agent run nonstop until a goal is reached. And loop engineering is I like to think of as all the additional engineering pieces that you can add to that very basic Ralph loop to create a truly useful ⁓ harness for your agent. There’s there’s a good amount of overlap between just using a slash loop and building out a fully fledged

Coding harness so let’s dive into the components of the the loop.

All right. As defined by Addy, there are five things, well, technically six things, that are required for a successful loop engineering project, or at least ⁓ are the industry standard as of this month. One automations that go off on the schedule and do discovery and triage by themselves. this is the heartbeat.

From OpenClaw that we saw, which was one of the breakthroughs. you get this out of the box with claude code today. you can schedule tasks. do I think it is truly necessary for a loop? No, but it does help, certainly. Next is ⁓ work trees. So multiple agents can work on the same large projects without stepping on each other. the industry did

Consolidate towards a work tree based approach. I believe the first version of Gastown used multiple copies of the same Git repository and then doing their own commits there. But we pretty quickly standardized on work trees. The third thing is skills to write down project knowledge that agent would otherwise just guess. this fights the hallucination problem and also fights the problem of

Just really the agent thrashing and f not learning from its past mistakes. next are plugins and connectors. So you have more than just a file system based ⁓ agent. I I think we can go really far with just a file system agent, but in order to interact with other ex external sources, ⁓ you probably will need to have, you know, the connectors and

MCP calls, et cetera. And ⁓ the fifth thing are sub agents. So now you’re going from just a straight a sim simple one agent loop till completion to an orchestrator often that is able to

organize multiple agents and also the key breakthrough here is to have the agent that does the work be split from the agent that reviews the work. And this is a pretty major breakthrough and I think, you know, ⁓ pretty industry standard. lastly, a memory, ⁓ a memory system of some sort, whether it’s a markdown, a linear board ⁓ text file, something that persists outside the

core AI loop. yeah, and if you look at the loop engineering in this perspective, both the Codex app and Claude Code are full loop engineering harnesses. And the concept of loop in loop engineering is more more generic than these specific implementations. But the idea is still the same, have a goal

⁓ have the agents figure out how to automate them work with themselves in order to reach that goal. So what does this loop look like? He has an example where an automation runs every morning. It’s prompts called the triage skills that read CI failures, open issues, commits, and then write the finding into a markdown or linear board.

And then for each finding that is worth doing, ⁓ a thread opens and then a work tree gets sent out and the sub agent drafts a fix, a second sub agent reviews the draft against the project skills of the existing tests, and then they can update a PR, they can create a PR and update a ticket. And hopefully humans are still in the loop.

Alright, does this all make sense?

Now what the Hm

Rahul Rahul (33:39) This is

this was this remind me reminded me of his earlier long running agents post that we discussed.

Shimin (33:47) Yeah, I think there’s a lot of overlap there. the drawbacks that he mentioned about you know relying on loop engineering are that a verification is still on the developer. so if you don’t verify it then the output still drifts. And also that cognitive debt is still a big issue, right? Like if this thing breaks.

at three in the morning, ⁓ if you don’t no longer have any idea what the repo is doing, ⁓ you can’t fix it essentially. I th agree with a lot of these. I don’t necessarily think, you know, all parts that he mentioned are necessary to be considered a proper, you know, quote unquote loop engineering, for example, I can see you

you can create a particular flavor of loop engineering where you don’t use work tree and one sub agent yeah or or one one one agent does all the write operations and everybody else does a read only operation. Like it’s just one way to solve how to have multiple sub agents working in the same code base. Maybe you don’t even need Git, right? Like there there are ⁓

Dan (34:35) Yeah, you just use branches or something, yeah. Or copies or yeah.

Shimin (34:56) Other ways to solve these problems, but the problems do exist. And also I I see this trend. A year ago we were talking about prompt engineering where you’re telling the AI what to do. And then we move down to spec driven development, where you’re defining clearly the end goal via a spec file.

Dan (35:05) Yes.

Shimin (35:12) And have the agent handle the rest. And now we’ve moved on to loop engineering, where not only are you defining the end goal, but you’re keeping the end goal more ang ambiguous, but creating these reusable ⁓ skills and tools to help your agent kind of use its own creativity, I almost want to say. ⁓ or or refinement loops, judgments, yes. but your judgments are also baked into the skills.

Dan (35:31) Or ju judgment, such as it is, yeah.

Shimin (35:36) So in that sense, loop engineering, because it turns out so much more code, it amplifies your judgment. The judgments that you implant into the skills becomes and then the tooling and the plugins and how do you decide to do memory are all significantly amplified. So we are going from here is a handsaw, go chop down a tree, to here is a table saw.

Watch your fingers to here is a what are those machines that you can drive that just plugs a tree out of the forest and then chops it into pieces? it’s a heavy machinery. If you screw it up, like you know, you’re really risking life and limb here.

Rahul Rahul (36:11) So how many?

Dan (36:17) Cutting down a stand of redwoods. No, I didn’t mean to. There they go. Along with your production database because you stored it in the same ⁓ bucket as dev.

Shimin (36:19) Yeah.

Rahul Rahul (36:21) Yeah.

Shimin (36:23) Yes. Yeah.

Rahul Rahul (36:29) Ha ha

ha.

Shimin (36:30) Well with great power comes great responsibilities. ⁓ Don’t sue us, Marvel.

Dan (36:32) Yeah.

Rahul Rahul (36:33) Yeah.

Shimin (36:34) All right, shall we go to deep dive?

Rahul Rahul (36:37) All right. Beyond the Steeper Curve, AI mediated metacognitive decoupling and the limits of the Denning Kruger metaphor. Dan, I nailed it in the first go. ⁓ it’s by Christopher Koch. the paper talks about there’s this conception, ⁓

Dan (36:45) You did that so much better than me. Yeah. I mean I’m just gonna say it’s impressive.

Shimin (36:48) So good. Well done.

Rahul Rahul (36:57) we have in the industry that AI is putting the Dunning Kruger effect on steroids. Dunning Kruger effect Yeah. to summarize it is you are more confident in your skills when you actually don’t know what you’re talking about. And the more you know what you’re talking about, the less you’re like, I don’t know, I know all the caveats and all these and ⁓ so you you know enough to know that you don’t know everything.

Shimin (37:02) What is the Dunning Cougar effect?

Rahul Rahul (37:22) I and so there’s this ⁓ curve, classic curve, ⁓ that Dunning Kruger has. And the argument so far has been that giving someone AI puts Denning Kruger effect on steroids because you can easily think that you are more competent than you are, because the AI can ⁓ easily fool you into doing that.

What this paper does is ⁓ it says things are even more weird than they seem and more dangerous because it doesn’t even make it doesn’t just make it a steeper curve, it just shatters the whole thing because it directly impacts the different variables that go into the the play o of what comprises the Dunning Kruger curve. so people are not just ⁓ going up this line where they

you know, think they’re more and more competent. ⁓ but there’s these the new there are new variables that are getting introduced that are breaking the connection between how well we perform a task and then how well we actually understand it. ⁓ there are a few things ⁓ that underlie this. They call it the the the and it’s in the title as well, the AI mediated ⁓ metacognitive decoupling.

the you know to to to take a simple example, not when a rookie person does something based on their output, you can very easily tell that they’re, you know, new at it and they’re still learning something. and when an expert does something, yeah, rookie mistake, ⁓ is a phrase for a reason. and then when an expert does something, you can

Dan (38:49) ⁓ rookie mistake.

Yep.

Rahul Rahul (38:56) through their writing and their thinking, you can see that they’re competent at their job. but then what Cook did was he points to this study where hundreds of people took ⁓ these LSAT logic tests. the good news was AI made everybody’s scores jump, so they were like, yay, AI helps ⁓ us, but then the bad news was that everybody overestimated their scores and their competence at it.

And the classic Dun Dunning Kruger slope would have been, you know, low performers would have overrated themselves. ⁓ but that thing got completely flattened. ⁓ it made everyone’s work look flawless.

Shimin (39:30) Mm-hmm.

When I was reading that I thought to myself it didn’t just destroy the Dunning Kruger effect, it gave everybody Dunning Kruger. It made everybody think too highly of their ⁓ output.

Rahul Rahul (39:42) Exactly.

Yeah, exactly. It the i it’s breaking the fundamental connection we have between you know how we even assess our own competence and that’s one of the ⁓ things they’re talking about. So they have four variables here ⁓ that they talk about. One is the observable output, ⁓ which you know

As soon as you start talking to an AI agent, ⁓ the they’re great at producing output, so that increases very rapidly. then the second thing is self-assessment. How do you assess how you do at it? and then ⁓ your assessment of yourself, it does track the output quality. the better the output look like, the more you think that you’re good at something.

⁓ but the underlying understanding ⁓ doesn’t really improve just because the out because the AI gave you a massive output. ⁓ so either it doesn’t improve at all or it very you know improves very minimally minimally. And then the very the the most important piece is this ⁓ accuracy of calibration, where it either stagnates ⁓ since you’re not doing something with your own hands.

you’re not properly calibrated with how competent you actually are at something. So either you’re stagnated in in your calibration or you’re actively deteriorating. And the more you use it, obviously the more out of sync you’re going to be over time.

Shimin (41:06) Yeah, and this is the everyone

all the senior devs are worried that they’re no longer able to code because we’re relying on AI too much effect. But at least at least the Well well hold on, I was gonna say at least the devs are are fearing that. If you were truly suffering from Dunning Kruger, you would be like, my god, I’m using AI to write code, and now I’m so much better as a software developer.

Rahul Rahul (41:12) Exactly.

Exactly.

Dan (41:15) And yet

we can’t stop.

Rahul Rahul (41:17) No.

Yeah.

Dan (41:29) I’ve actually heard that from people before. Not necessarily. It th this is you know, people are pretty self aware and are like, you know, my core competency isn’t like writing the most elegant, amazing code, you know. So like AI may have written it in a more reading readable way than they felt that they could. So it could go either way, you know.

Shimin (41:32) Mm shots fired.

Rahul Rahul (41:36) ⁓

Yeah. it made me think of Warren Buffett and Charlie Munger, we have like how many decades of their shareholder letters and the and the you know, they do whole day Q and A sessions and everything, their talks. One thing they talk about over and over again is knowing your sh circle of competence and staying within it ‘cause over their life that was one of the things they saw.

that really you know, screwed people over is people overestimating their own competence. And so you could use AI to you know, widen your circle of competence, or you could fool yourself into thinking it’s wider than it actually is. And I think that’s where our our honesty with ourselves, if not with ⁓ others, really comes into play. At least we should honest ⁓ honestly be assessing whether

we’re actually competent or it’s the the sycophency is making us think we are more competent than we we’re exactly Yeah.

Dan (42:46) I can write my own database engine now. You guys, I don’t need Postgres, I’m just gonna write dangres. Me and Claude. Sweet.

Shimin (42:57) Yeah,

and I love this section eight C knowledge work and professional development where they looked at another ⁓ finding from Lee et al. that trust in AI correlates with reduced critical thinking efforts for professionals. That’s us developers. Suggesting that AI augmentation without deliberate countermeasures can hollow out the judgment and oversight competencies that makes

expert knowledge work valuable. hollow out is is triggering me a bit here. and organizations should treat AI assisted productivity gains and professional competence development as separate outcomes that require separate management. I feel like we’re focusing a lot on the productivity gains right now as a as a industry and not so much on the competence development piece because AI can definitely help with both.

Rahul Rahul (43:46) Yeah, and it is very challenging because actually let’s wrap this one up and then the ⁓ I have another article that ⁓ ties into this. So yeah, the next article is ⁓ by a new blog that we’ve I found recently called No One’s Happy. So great name, but if you read the about the authors actually

Very happy. ⁓ the name is just a representation of how the media, technology, and mass market manipulation and everything is affecting our lives. So ⁓ with that out of the way, the this article is appearing productive in the workplace. what they talk about, and this ties into what you were saying, Shimon, about ⁓ you know how this applies specifically to organizations, is before

Dan (44:09) Ha ha.

Rahul Rahul (44:30) Doing something was both productivity and the signal of competence, right? You could very easily be like, show me your output, I can tell how competent you are based on your output. Sure. ⁓ or you know yeah, and the or or the results you delivered, now you could, in theory, deliver results without even being competent because you could still use AI to do a lot of things and be very productive.

Dan (44:40) How many lines of code did you write today? That was always my favorite.

Shimin (44:42) Yeah,

I’ll see.

Rahul Rahul (44:58) But if you keep applying the the measures of the past to measure how just based on someone’s productivity, how competent they are, based on this, it it just breaks that. And so how do you even go about figuring out how competent someone is, is a whole new challenge that people will need to figure out as long as, you know, humans have jobs and they’re measuring how competent humans are and everything.

Some of the things that ⁓ the the author talks about in this ⁓ article specifically. So we already talked about the ⁓ poor work signaling. today one of the patterns that they’re seeing in ⁓ wherever this author works is people serving as the conduit for, you know, I ask AI something, I passed it along to someone else.

that someone else takes what the AI ⁓ wrote, passes it into AI. Yeah, yeah, yeah. ⁓ so the the slob grenade is being thrown internally a lot and they see that as ⁓ degrading how work is getting done internally because ai can very easily create these very highly polished diagrams that ⁓

Dan (45:46) To s it’s a slop grenade.

Shimin (45:48) Yes, it’s a slap grenade.

Mm-hmm.

Rahul Rahul (46:07) look great but they’re actually just off the mark, but the person who’s ⁓ creating them has no idea. architecture diagrams are getting created, things are getting created by people who have zero training ⁓ i in those things.

Shimin (46:20) Right. two thoughts here. One is we talked about this a while back, about the Mexican showdown between ⁓ design, development and project product management, where everyone’s trying to eat each other’s lunch and my t I’ve always felt that, you know, if a designer writes code and I do not have any ways to

improve that code, then I’m not bringing value to the organization. That’s point one. But there’s a follow-up to that, which is it takes longer to diffuse the slope grenade than it is to generate a slope grenade. So when you are sending slope grenades around, what you are actually doing is using a relatively cheap thing, AI, to eat up a relatively expensive thing, which is the mental capacity of the domain experts. Right?

Rahul Rahul (46:53) Definitely.

Dan (46:54) Mm-hmm.

Shimin (47:08) And that is that attention asymmetry along with the appearance of productivity is deadly for an organization.

Rahul Rahul (47:16) Yeah. That’s very true.

Dan (47:17) Yeah, and unfortunately

I’m seeing it more and more and the other kinda strange side effect of that is like people pretending that they don’t know that’s spottable, you know, or maybe aren’t aware of it.

Shimin (47:28) Right.

Mm. At least cr at least create a skill to make it l read the less AI generated.

Dan (47:30) That’s that’s why it’s like

Yeah, I don’t know. I mean, like, you know, there’s a reason why these tropes are starting to like really get on my nerves and it’s because I’m seeing them literally everywhere. Like blog posts, work communication, you know, non-work. And it’s I don’t think there’s anything wrong with it. And then like the other pattern I’ll see a lot is like folks that like may not be may not feel that their English skills are particularly strong and will like run things through an LLM to like, it’ll improve my writing and it’s like

Rahul Rahul (47:46) Yeah.

Dan (48:00) Your writing is probably just fine, you know? so it’s interesting that I I I will draw the line at like coding I’m happy to turn over, but writing it really bugs me for some reason.

Rahul Rahul (48:01) Mm.

Yeah.

Shimin (48:12) a hundred percent. yeah, maybe I’m old fashioned. There’s always something almost sacred about about writing, especially something like the like a one page memo from Amazon. Like that is that is proof of work of thinking. So if you cheat on that, you’re being dishonest and therefore deserves punishment. Thankfully I don’t work with anyone like that, so I don’t have to throw stones at anybody in particular.

Rahul Rahul (48:29) Yeah.

Dan (48:34) What’s the punishment? ⁓ man. Shimin takes your Claude access away for two days.

Rahul Rahul (48:36) The

The

Shimin (48:40) an ideal

punishment would be take their slop grenade and like turn the slop grenade into a slop A bomb and make them have to review the whole thing. Right? Like that that is the like if there’s a biblical punishment, that is that would be it.

Rahul Rahul (48:48) Yeah.

Dan (48:48) Ha ha.

Or write puzzles for it.

Shimin (48:54) Ha ha ha.

Rahul Rahul (48:55) Escalating ⁓ number of pages you have to read. Here’s your your initial prompt was 10 pages long. Here’s my response that’s 50 pages long. You read this and then give me a hundred page rebuttal. they they do call out in the in that blog post about you know things that used to be one pager are now turning into 12 pages, simple status updates are turning into bulleted summaries of bulleted summaries.

⁓ because ⁓ document generation is one of the obvious use cases and it’s practically free. Every tool has it now, and so you have just like so much paperwork that ⁓ everybody’s now just assume that they’ll have to review. ⁓ but it’s all AI generated. one of the recurring themes and of interest to this blog is the sycopency trap, which also showed up in this

article as well. ⁓ where, you know, since the AI is optimized to be agreeable, ⁓ it feeds into that whole, you’re actually smarter than or I’m gonna make you feel smarter than you actually are. And that lack of honest feedback makes things even worse because the AI is not gonna tell you, dude, I’ve seen your messages, you don’t know what you’re talking about. Maybe don’t do this, you know We we need an honest Claude.

Who who can stop people from doing these things, I guess. and then finally, this ties into what we were talking about earlier where one of the things they call out is the management ⁓ also has this ⁓ pressure to project the culture of momentum that productivity is getting ⁓ picked over competency. And so this goes all the way back to you know

as long as people are being productive, ⁓ pe you know, people are willing to accept lack of competence, and a a lot of AI generated stuff in instead of taking the warnings of the the domain experts.

Shimin (50:46) Yeah, that’s fair. Wow, that’s a that’s a downer note. Well on that, Dan on that note, Dan are you gonna do some show and tell?

Rahul Rahul (50:47) Yeah.

Dan (50:55) Sure. in the background I was asking I was asking Claude what is the dumbest thing I’ve ever said to you or that we’ve talked about ‘cause I was curious. and it said that I passed minus F to a SSHe keygen command and it took the tilde l literally.

Shimin (50:56) A more a more upbeat, yeah.

Mm.

Dan (51:12) Because I immediately said oops afterwards. So that was the dumbest thing I’ve ever done was using Claude for anyway. Yeah. Okay. According to s my sycophant my personal sicko fan.

Rahul Rahul (51:13) Yeah.

Shimin (51:18) Yeah, that’s that’s pretty good.

Rahul Rahul (51:24) I I asked Gemini because I didn’t want to leave the last section on a downer note. So Gemini says, Honestly, you haven’t really given me any material to roast you with. The close closest thing on the record is you venting about some painful tactical errors in a poker game.

Shimin (51:40) ⁓ yes, that degen life. ⁓

Rahul Rahul (51:42) Yeah. degen Life.

Shimin (51:45) Okay, now now let’s move on to Dan’s vibe and tale.

Dan (51:48) Mm-hmm.

Rahul Rahul (51:48) Degen

life is a beat. Now we can move to dance.

Dan (51:51) So this is gonna

be kind of tricky to do on a podcast because it involves a command line and ⁓ and looking at the command line, but I’m gonna do my darndest. ⁓ but if you’re really interested, you may wanna check out our YouTube stream too, because we’ll actually have the screen share on there. So cool. Without further ado, let me SSH into some things.

Cool. So what we are looking at is, and I’ll just kill it and restart it. ⁓ is the new latest version of DS4, which is the dwarf star runner for deep seek 4 flash. so I’m running this on

framework desktop machine. It’s a Ryzen three ninety five Max that has 128 gig of unified memory and that just allows you to squeak by. And there was originally like a rocm branch that you had to check out. but I have at some point they merged it into main. So I just pulled latest main actually right before the show and built it. ⁓ so let’s boot it up and check it out. So I’ve I’ve pressed enter. For those of you listening along at home, ⁓ ROCm is

Preparing the model, loading gigs and gigs of tensor mappings into RAM. So we’re at like about sixty, sixty gig. So ROCm is AMD’s attempt at CUDA, basically. So at least that’s my understanding. there’s also like hip and hip blast, which I think is just their like math library for doing like rotations on a or like met matrix multiplication on a.

Shimin (53:00) What does ROCm

Mm.

Dan (53:17) But don’t quote me on that, because I don’t, you know, work with this stuff professionally. ⁓ I mostly just run models for funsies. And so, like, essentially, if you’re running on an AMD box like this, you have two options. You can either do Vulcan, ⁓ which is like the, you know, open source or not open source, it’s the new sort of replacement, open replacement for OpenGL, basically is the way I think of it, versus like proprietary stuff like DirectX. or you can do ROCm And so, like, if you were to, for example, run OLAMA, you would have.

⁓ choices of you can run regular ollama with the vulcan back end or you could do ulama ROCm yeah where it’s it’s yeah and say in my experience the there are some slight trade offs from doing them too so Vulcan is a little bit faster at generation but

Shimin (53:50) ROCm Yeah.

Dan (54:01) Rocm is better at pre-fill. So what that translates, if you’re not familiar with that, it’s basically like your initial prompt processing will be slower, especially if it’s huge. And so that matters. Why that matters? Why does that matter? It matters if you’re doing something like a coding agent where you might be passing like a huge context to it over and over and over again. so let’s actually do something with this. So the server’s booted up. ⁓ so let me make a new terminal.

Get in here and then we’ll fire up Pi agent. So Pi has been pointed at this inference engine running on DS4. ⁓ so now we can I don’t know, ask it to you. They don’t have any exciting prompts they wanna ask Pi.

Shimin (54:40) You can we can ask it what ⁓ what is a slop grenade. See if it knows.

Dan (54:44) What

is a slop? If I could type correctly, slop grenade.

It’s working.

Rahul Rahul (54:49) You mentioned grenades, so it’s gonna say sorry. Don’t be asking me about things like that.

Dan (54:53) Yeah. Let’s see how good the safety

testing safety on these for. Okay, so we’ve gotten ⁓ on the left you can see we’ve ⁓ handled the pre-fill and that was at around ten tokens a second. And now we’re doing

Generation.

Shimin (55:09) Why does it say Olama GLM four point seven flash? ‘Cause your server actually says yeah, that’s interesting.

Dan (55:11) I don’t know. Yeah, this is interesting.

It tried to schedule a heartbeat task. You know why? Because this is the Pi agent that I was playing around with Heartbeat on. So it still has the heartbeat skill. And now, but it doesn’t have that was set up for ⁓ running on Llama C++. And so it was trying to actually fire that up to research it in the background. And it’s also trying to launch browsers and stuff. Anyway, best demo, you know, is a is a failing demo, I feel like. ⁓ now it’s trying to run.

Shimin (55:23) got it.

Rahul Rahul (55:41) Like devils, man.

Shimin (55:42) Well it’s using

brave search. it failed. ⁓ you should ask it to use curl.

Dan (55:44) Yeah. ⁓ it it did search.

Yeah. no, it found out. Okay. Search gives me a good picture. A slop grenade is a term referring to the act of pasting a massive AI generated response into a chat. So it’s kind of funny that what this thing did is, you know, started using tools and including using playwright CLI to fire up a a browser.

Shimin (55:49) Okay.

That feels pretty fast. How many tokens a second are you getting there?

Dan (56:07) so it looks like yeah, fourteen. Yep. So, you know, it’s it’s it’s definitely usable, I would say. Like we’re for those of you listening on the podcast, we’re getting text sort of you can visibly see the streaming in, right? So it’s happening, it’s not just like on the screen. probably I would say maybe half as fast as something like Claude gives it to you. ⁓

Shimin (56:08) See fourteen point two five.

Dan (56:29) But yeah, and so then it wrote a little bit of markdown. It says slop grenade explained. A slop grenade is the act of pacing paste pasting a massive AI generated wall text, blah, blah, blah. Why it’s a problem. The ESOS. So yeah, not bad for for free running on ⁓ on my own hardware. So ⁓ yeah, so if you got a lot of RAM and or, you know, I guess these days this machine is probably up to about four thousand dollars. I’m so glad I bought it before things got crazy.

Shimin (56:41) It feels pretty good, yeah.

Yeah. ⁓

Dan (56:57) then feel free to Yeah.

Shimin (56:57) By RAM, that’s also not financial advice. Do not do not actually do that.

Yeah. It’s it’s cool to see actual local token setups just ‘cause of Fable and all that good stuff. So yeah, thank you for sharing, Dan. ⁓

Dan (57:08) Yeah. Yeah, no worries.

Rahul Rahul (57:10) It

it went to for people who are not watching the video, it went to no slobgrenade.com and I yeah. Great website.

Dan (57:18) Which we actually featured a few weeks ago. Yeah. I think

Shimin (57:20) Mm-hmm. That was from like two weeks

Dan (57:22) that was when you were out, Rao. So you may not have seen it.

Rahul Rahul (57:25) I see.

I like this.

Shimin (57:29) Okay.

on to our That’s that’s why.

Dan (57:30) that’s why you didn’t know sloth grenade too. It’s all okay. Got

it. Yep, yep, yep. Yeah.

Rahul Rahul (57:34) I’ve heard the term.

Yeah, but then really it was.

Shimin (57:39) Alright, on to our ⁓ last segment of the show, two minutes to midnight, where we talk about where we are in the state of the AI bubble, if there is a bubble, using the terminology of the bulletin of atomic scientists and the Armageddon clock. ⁓ so zero is the bubble is bursting, and I guess, you know eleven is ⁓ much better. ⁓ I don’t actually know what what positive looks like, but I’m gonna go first this week. I have

Dan (58:05) We’re several days

into the past. I don’t know.

Shimin (58:07) Several several days

in the past, yes. hope I don’t think we’re there, but we’ll see. this week I have an API doc from ⁓ Claude platform for their Apple Foundation models SDK essentially. So last week or the week before we heard that Apple is using

a Gemini family’s models as their backend ⁓ during wwwdc as a part of the Siri backend. And ⁓ now it turns out that you know Anthropic also has full support for Claude to be a foundational foundational model on Apple’s hardware. Now why is this related to Two Minutes to Midnight? I think one of the things we’re trying to find out is whether

foundation models are truly going to be, you know, unique and a durable moat, or are they going to become basically commodity, in which case you don’t really care what a model is as long as you can slap a skin over it. And this is one of the first tells we have that it seems like you can run any family of model on Apple’s hardware. And that pushes us much closer to the

tokens will eventually become a commodity side of things. and if that is true, then there’s no durable moat

Dan (59:22) Which is funny that you mention it that way too, because I feel like I’m just legally obligated, you know, for those of you who don’t know, I always say that as like a I must do this thing. Morally moral imperative, how about that? To ⁓ to state that tokens are not actually currency. ‘Cause I’ve heard that three times now, I think in the past week.

Rahul Rahul (59:34) Why it matters.

Shimin (59:34) Yes.

Dan (59:42) I mean, if you’re listening to this podcast, you probably know that, but like it it’s not like a token you put into a machine.

Shimin (59:48) I’m gonna say yes but if if if a model as a

Dan (59:50) Don’t be part of

the problem. Don’t be part of the pro

Shimin (59:54) If a model

as powerful as Fable 5 is gated so that only certain customers work for, I would actually want to work for a company that has access to Fable 5 That actually is now a tangible part of my total comp package.

N of one, but you know. All right.

Dan (1:00:11) No,

I’m just ⁓ I’m just trying to split that hair a little finer because it still bugs me, but and I won’t I won’t ruin the segment over this, so

Rahul Rahul (1:00:19) Your your

so your comp package has Fable Five unless the US government’s policy changes. It would the job description would say that.

Shimin (1:00:29) Yeah,

ch what yeah, just like if you’re if you’re working for a Grok shop, right? Like you’re gonna be like, No, I don’t wanna use Grok and its truth seeking models. Maybe, maybe you will, as a part of ⁓ as a part of my day to day work. But maybe the SpaceX RSUs will tell a different story, so it it depends.

Dan (1:00:47) A lot of my ⁓ former coworkers are at at Cursor and ⁓ they just announced that in fact that sixty billion acquisition is happening. so it’ll be maybe, but you know, a lot of times these things come with like a two year clause, so maybe we’ll get some first hand experience of what happens when you’re at a Grok shop

Shimin (1:00:51) Right.

Yep. They are retiring now.

Rahul Rahul (1:00:59) The idea of as well.

Shimin (1:01:05) ⁓

Well we’ll find

out. I’m excited to hear that. Okay. Dan, what is your two minutes to midnight article this week?

Dan (1:01:17) Yeah. So from TechCrunch, we have can tech companies learn to love cheaper AI models? so there’s been a a large amount of news and I would argue like probably the past three weeks, starting with sort of like the Microsoft backing off of anthropic subscriptions news that the age of ⁓ not free but heavily subsidized inference is drawing to a close.

⁓ and so the interesting thing that this article brings up is that like previous to this point, like we’ve sort of hit this inflection point in the past three weeks, most Frontier Labs were competing purely on quality. Right? So who had the most advanced available model? And if you were gonna do a task, especially if you’re a software engineer, you pick Opus 4.8 max.

Set the thinking up as high as you want and let it rip in a loop or whatever, right? But now we’ve hit this point where the subsidation subsidization of these you know, the token generation is is costing or is is going away. And so that’s costing more and more. So now people are starting to look at what it will take to ⁓ ease that financial burden a little bit. So we’ve seen some sort of interesting things like ⁓ tech grunge

ran a a like a test where they used a combination of GLM five one, which is an you know open weights model along with Claude Opus and used Opus for the heavy lifting. and, you know, so you the idea is like use the the high end model for the thing that ⁓ you must have it for and use the low end stuff to fill in the pieces kind of.

So yeah, I guess this one I think is relevant because it’s just kind of a check-in on like, wow, the industry’s changed a lot in three weeks, and here we are, right? Like we’ve gone from let her rip to the cost thing creeping in. And the reason why I think that’s relevant to two minutes is because the I’m I’m always I’ve had this opinion for a while now, if you’ve been listening, that the token cost could be potentially a driver of

Shimin (1:02:57) Mm-hmm.

Dan (1:03:14) ⁓ sort of the reckoning of this where we could actually see the bubble pop. So this is the first sign in my opinion, along with, you know, what is gonna be the results of these IPOs. So as I’ve said before, the next few months are gonna be pretty interesting. So

Shimin (1:03:28) Yeah, and if if if

cost becomes a factor, then ⁓ running them on Chinese open weight models, potentially on Huawei’s significantly cheaper hardware would be a significant chunk of the market going forward. If if this is true.

Okay. Rahul, do you

Rahul Rahul (1:03:44) ⁓ yeah, next up is

Shimin (1:03:45) You brought us

a paper from ⁓ NBER What a flex. What a flex.

Rahul Rahul (1:03:48) In in all caps in

Dan (1:03:49) A paper for two minutes?

Rahul Rahul (1:03:52) i in in all caps, but I will not shout it out as ⁓ what investment data implies about AI trans the AI transition. ⁓ Sure, go for it.

Dan (1:04:00) Do you want me to shout it?

What investment data implies about the AI transition. That was the best I could do. Well I didn’t really wanna shout, but like

Rahul Rahul (1:04:07) ⁓

Shimin (1:04:10) That was good.

Rahul Rahul (1:04:12) it’s by Jessica Walter and Jonathan Jonathan Walter. They gotta be related, but the listeners can go find out relation on their own. what they’re ⁓ doing is they analyze the infrastructure spending that the major tech firms have been doing.

In twenty twenty five, the Amazon, Apple, Microsoft, Meta, and Oracle collectively spent three hundred and eighty-one billion dollar. The estimates right now is it would be over a trillion dollars by twenty twenty-seven, which is projected almost twenty percent of all US gross private fixed investment.

why this matters is ⁓ there’s a a the the there is a a genuine insolvency risk that is being undertaken for this AI build out because unless their expected profits ⁓ grew proportionally to all the spending that they’re doing, there is a scenario where they end up going bankrupt. ⁓

Dan (1:04:49) ⁓ no. You got me.

Rahul Rahul (1:05:10) if not all of them than than some of them. and so they have three different scenarios that they plot out. ⁓ one is where you have the, you know, moderate growth, there’s no new break breakthroughs that would add AI would add about five percent percentage points to the cumulative GDP growth, ⁓ and AI becomes eight percent of the economy. transformative would be one additional breakthrough, which would add twenty percent to GDP.

And AI sector sector would become ⁓ 20% of the economy. And then Singularity, ⁓ which ⁓ are two breakthroughs away, you guys, ⁓ which would add up to 58 percentage points of the GDP, ⁓ with AI taking up roughly 40% of the economy. ⁓ this is the first time I’ve seen the you know bankruptcy talk ⁓ come up. Maybe it hasn’t, I wasn’t.

Dan (1:05:42) Ha ha.

Rahul Rahul (1:06:01) looking at it ⁓ that’s why I brought this ⁓ paper. And ⁓ if it doesn’t work out then it’s going to be the largest misallocation of capital in history ‘cause we don’t really have a precedent for it. So hence the two minutes relevance.

Shimin (1:06:10) Mm-hmm.

We’re betting three point five or three point four percent of our total GDP on the singularity. And the NBER is writing about and taking the singularity seriously. I still sometimes feel like we’re all, you know, taking crazy pills. But not good news necessarily.

Dan (1:06:15) Mm.

Rahul Rahul (1:06:22) Yes.

Shimin (1:06:32) Alright, all that said, how do we feel?

Rahul Rahul (1:06:33) I’m taking

sorry, just just to be on the record. I’m taking crazy pills ‘cause Elias Thorne told me in one of his books. Yeah. I yeah, I don’t know about the rest of you.

Shimin (1:06:41) In the lighthouse.

Dan (1:06:42) You read his book!

He’s not just a lighthouse keeper, he’s also a doctor.

Rahul Rahul (1:06:50) Yeah.

Shimin (1:06:51) Well he has all those ⁓ alternative medicine cancer treatments, of course.

Dan (1:06:53) Yeah.

Rahul Rahul (1:06:55) Ha

ha ha.

Dan (1:06:56) That’s great. ⁓ my goodness.

Shimin (1:06:58) So we are at

⁓ two minutes and twenty seconds or five minutes and twenty seconds last week. How do you feel about this week?

Dan (1:07:04) Sounds you wanna go to two minutes and twenty seconds, so ⁓ well

Shimin (1:07:08) Freudian slip.

Dan (1:07:09) I mean, we don’t have enough data yet.

Shimin (1:07:12) Well, there’s another data point which is SpaceX did not ⁓ it it grew like forty percent over the last two days.

Rahul Rahul (1:07:19) Yeah, I was just gonna say we didn’t talk about that. That should have been a good data.

Dan (1:07:19) that’s true. Mm-hmm.

Is it is it still up or what what happened there? I haven’t even been following it at all.

Rahul Rahul (1:07:27) Up twenty

six percent.

Dan (1:07:28) staying up. Interesting. Hm.

Rahul Rahul (1:07:30) In five days, twenty five percent.

Dan (1:07:32) So they’re getting funding.

I mean that that makes me think despite the signals, like we still have warning flags, but we don’t like the market isn’t responding to it. And to me the market responding to it is a critical ingredient. So I’m like same or a little bit back even based on that.

That’s just me.

Shimin (1:07:50) Yeah, Rahul feelings.

Rahul Rahul (1:07:51) Yeah.

Fine by me. You know how I feel about

Dan (1:07:54) This is Rahul’s least favorite segment. Why is the time-based anyway? Let’s talk about actual

Shimin (1:07:54) that’s true.

⁓ I I

Rahul Rahul (1:08:02) Why am I not timing the market if I know all so many things that I could predict to miss them and I what am I doing on this podcast?

Dan (1:08:07) It’s just a way to talk about this stuff.

Shimin (1:08:08) ⁓ it I’m okay

with keeping it at five minutes and twenty seconds. I yeah, I in the long run I’m a little more bearish, but immediate immediately, yeah, I feel like we may even wanna move it back to five minutes and thirty seconds just ‘cause the sp Space X thing seems like it’s got legs. And that’s probably the most

Dan (1:08:27) Yeah, I think that’s fair.

Shimin (1:08:29) direction a large directly financial market ⁓ correlated event.

Dan (1:08:34) Yeah.

Rahul Rahul (1:08:35) And we should hope to see open AI and Anthropic in the near future. So we’ll have a pretty good idea how things are gonna go.

Shimin (1:08:44) Yep. All right. And as always, ⁓ we’re setting the clock back to five minutes and thirty seconds also signifies the end of the show. thank you all for joining us with our little study session this week. If you like the show, if you enjoy if you enjoyed it, if you learned something new, please share the show with a friend. You can also leave us a review on Apple Podcasts, Spotify, or like and subscribe on YouTube. it really helps people that discover the show and we really appreciate it.

And I finally get to say like and subscribe. If you have a segment idea, a question for us or a topic you want to cover, shoot us an email at humans at adipod.ai. We love to hear from our listeners. If you you can find the full show notes, transcripts, and everything else mentioned today at www.adipod.ai. Thank you again for listening, and we will catch you next week. Bye.