I've tried. It doesn't seem interested. Or capable. I theorize that's because it has no payoff function for "code that's easier to change". It also doesn't know about automated refactorings, so it laboriously makes changes (including lots of mistakes) that could have been done in the IDE in a single menu action.
When I started using the genie in my workplace, I also realized I was too enthusiastic about letting it add features. Even if I required it to follow best practices and add unit tests, I got diminishing returns over time. Now I’m approaching it as you described, asking for small, well-defined changes. I think of myself as writing narrowly-defined user stories and then doing a combination of refactoring and acceptance testing. This process is slower, but I can achieve much more than I ever could un-augmented. I’m still very excited about the future of augmented intelligence
From my experience there is a general pull in the latent space to generate responses that are selfcontained. This comes most likely from optimisation for coding benchmarks in the finetuning process.
I‘m trying to only work on the abstraction of specification and do no longer write a line of code. If the LLM returns bad results, I refine the specifications and retry until it works. Some of my specs are down to pseudo-code, to give a good guidance. I do seperation with interfaces whenever possible. So the LLM only needs the specs (requirement + design) and the interfaces to implement. Maybe an example implementation if avail.
I've been working on set of guidelines that I can give my genie clear guidelines. I've yet to get it to go from start to finish without getting out of control. But I did get it to write a terraform project for 1 very narrow focus without making a mess.
I have to prompt it explicitly to refactor, but as long as i keep on top of it and make it back up with it goes crazy, it'll do TDD of a sort with me.
Do you give your genie space for refactoring? 🤔
I've tried. It doesn't seem interested. Or capable. I theorize that's because it has no payoff function for "code that's easier to change". It also doesn't know about automated refactorings, so it laboriously makes changes (including lots of mistakes) that could have been done in the IDE in a single menu action.
Excelente point 😅 maybe an MCP tool idea 💡
When I started using the genie in my workplace, I also realized I was too enthusiastic about letting it add features. Even if I required it to follow best practices and add unit tests, I got diminishing returns over time. Now I’m approaching it as you described, asking for small, well-defined changes. I think of myself as writing narrowly-defined user stories and then doing a combination of refactoring and acceptance testing. This process is slower, but I can achieve much more than I ever could un-augmented. I’m still very excited about the future of augmented intelligence
From my experience there is a general pull in the latent space to generate responses that are selfcontained. This comes most likely from optimisation for coding benchmarks in the finetuning process.
I‘m trying to only work on the abstraction of specification and do no longer write a line of code. If the LLM returns bad results, I refine the specifications and retry until it works. Some of my specs are down to pseudo-code, to give a good guidance. I do seperation with interfaces whenever possible. So the LLM only needs the specs (requirement + design) and the interfaces to implement. Maybe an example implementation if avail.
Breathing was the perfect analogy for this.
Inhaling: AI is doing just that - if you let it.
Exhaling: Humans have the capability of doing this.
I’m curious if we’ll reach a point where this is true for AI.
I’d argue humans are capable of just inhaling as well.
In this sense, we have laws and regulations for those who only inhale.
Breathing was a perfect analogy.
I've been working on set of guidelines that I can give my genie clear guidelines. I've yet to get it to go from start to finish without getting out of control. But I did get it to write a terraform project for 1 very narrow focus without making a mess.
I have to prompt it explicitly to refactor, but as long as i keep on top of it and make it back up with it goes crazy, it'll do TDD of a sort with me.