So this makes sense for a standard SaaS app - but given that models in general perform much better with low context window usage, it probably also means that Fable is still significantly better at 'frontier-level tasks' -- hard research problems, complex geometric rendering algorithm optimization, etc., no?
It doesn't seem like there's been a precipitous drop in resources compared to the decades of requests and warnings that have led up to this point. So what's different now, if not resourcing?
There hasn't been a preciptious drop in outcomes either. There have been statistically significant drops in average test scores, but the large number of students who take those tests means that even small differences can be statistically significant. Generally, the average test score just fluctuates within a few percentage points over the long term. The differences between individual students are much larger. If you pick two random students in a year and compare their scores, they'll likely be much farther apart than the average scores of different years.
As a corollary, the variation that people personally experience at small scales (e.g. high-school teachers comparing the various students they encountered throughout their career) is dominated by changes in class composition. Some years, there are just randomly more bad students than in others. When the students seem to be getting worse over time, the teacher might attribute this to societal decline; when the students seem to be getting better, they credit their skill at teaching instead.
Thus things are constantly getting worse and the sky is falling, yet somehow it never makes contact with the ground, and when you compare with ancient records, it's more or less where it has always been.
I'm not sure which precipitous less than a decade drop you are referring to, but I would be inclined to think, in the last decade, a period of social isolation and absence of education might have been a factor.
Yes, but you can get away with it when you keep 51% of the ultra-high quality brain power of society in bondage. If you want to replicate American greatness under conditions of free competition, you must ~triple the wage. People do not think through what America had in its public schools in the postwar period and expect good results when wages have fallen behind even nursing. Ask your preferred AI to compare nurse:high school teacher:dentist:physician between the 50s and today, keeping in mind that high school teacher pay was grossly suppressed by the bondage of women. The teaching staff of American greatness and economic dynamism was ultra-highly educated women who were paid basically nothing. Teachers are paid in a much lower proportion to e.g. physicians - another hightly trained service, than they were in the 50s. The difference is that women can be physicians. Educator wages and thus competition for them is infinitely too low in contemporary America. If you want to bash teachers, I'm fine with that, the fact is we get what we pay for.
"In my experience, AI still drifts from what I meant it to do on anything bigger than building a widget."
I've had code bases with tens of thousands of lines of code built from scratch that I hand-reviewed every line of and worked with the AI to improve, and haven't had this issue. I feel like a significant part of this is due to an involved /plan stage -- going back and forth on building out a plan for what you want the AI to do involves surfacing the assumptions that you would have called drift if you asked them to implement it directly from your prompt.
Once the plan has been refined and is what I want it to be, getting it to implement everything in TDD style has for the most part given me 100% working code, as I wanted it to be, without issues. It definitely helps that I'm a principal-level engineer with extensive architectural experience -- but if you're able to tell the AI in detail what you want, have it ask questions for clarifications, and read through a plan before getting it implemented, and have a solid testing plus manual qa process (automated by chrome devtools mcp) in place, I've find that you can one-shot complex features, rewrites, and even not-insignificant applications that would have taken days to write by hand in a few hours.
Depends - using Sonnet here and generally it should be as you say: plan would produce the result.
Still Claude will sneak things in - in my recent plan, for example I had defined, per acceptance criteria what colours the statuses should be: green for live, blue for sold, grey for anything else; it changed this to: green for live, orange for in progress, blue for sold, red in demolition, etc. When pressed why did it to this, it was unable to explain why. This is with a plan where AC were explicitly provided from the task in Given/When/Then format and were to be adhered to strictly. I've caught this within planning, but I shouldn't need to be doing this.
Even in standard prompts where I tell it "Change this label from X to Y", it ended reordering the tabs unrelated to ask. Again I was not able for it to explain why - it was so abrupt. And it was in fresh context, without any pollution on what I expect it to do.
I also noticed a different behaviour regarding skill; today and yesterday it would not be following skill guidance at all ie: skill writing skill - I'd have to explicitly tell it to test skills after writing them, when this is a behaviour expected by default. Similarly with other skills - knowing that it should have done something per skill guidelines and it not doing it at all. This is new behaviour that I've not seen a week ago.
There are certainly domains where AI is not so effective, but at this point I would agree that at least in terms of web development if you can't get effective results from agents at this point it is a skill issue. That skill can be learned, if you recognize that learning is part of the solution. I do think prior experience in product design, specifications & business analysis as well as engineering leadership are all extremely helpful. Its about putting the agent in a box so small that it really can't screw up; but its also about being able to review design and code rigorously - to see around corners and anticipate possible weaknesses etc. There is really nothing I have to do when working with an agent that I haven't already been doing for decades but it seems to me that a lot of developers have never found a single bug while reviewing someone else's code.
If you take the chromatic scale and then swap every other pair of notes on opposite sides of the circle, it yields the circle of fifths. You'll notice that on the circle of fifths notes that skip a step are a whole tone apart in the chromatic scale.
Although there have been some claims in these comments to the contrary, harmony is particularly mathematical. Symmetry and the breaking of within the integers mod 12 form the foundational principles of harmony.
Abstract Algebra, Combinatorics, and Discrete Mathematics are all definitely worth the squeeze; and incidentally something that could easily be taught to middle- and high-schoolers with the right examples.
Many artists in fact do exactly this -- they do not make the art themselves, but instead imagine and manage the 'art project' and delegate tasks to other artists that work for them. Leonardo (Da Vinci) in fact himself did this with much of the work required for his paintings, although the most crucial parts he left for himself.
They are slant rhymes of one another. B and M are phonetically nearby, as are ia and ih and ao and ou. In no way like hacker news and cake her knees -- but more like hacker news and hagger moos ;)
reply