A while back when one of the popular models came out it blew all the others away in benchmark tests. That got my coworker and boss super excited, we started coming up with different ways AI can help us. Thankfully as I pointed out, our software is proprietary and super secret, and all it took was a couple Google searches to find out a lot of those companies leak data like crazy and AI will just tell other people your secrets if they ask in the right way. So we needed to run our LLMs locally, but for that we’d need some beefy specs. I did the research, wrote a neovim and sublime plugin to integrate our local LLMs in a ‘copilot’ kind of way. My boss ordered my coworker and I whatever the new macbooks are with 128gb of memory to fit our lovely AI models, bought me a desktop tower and a few GPUs. Then I went on a 3 month paternity leave, came back and have heard nothing about AI since aside that my coworker switched to vs code to try and use continue but got frustrated, switched back to sublime and doesn’t use AI anymore.
So yeah, AI got me 2 new maxed out spec machines and 2 weeks of fuck around time writing plugins that were not nearly as complex as everyone thought, just because AI was involved.
For real though, I’ve tried quite a few times but 9/10 it fails me and I end up spending more time messing around with prompts than it would’ve taken me to do the thing. Occasionally when I have a mile long error message or something super obscure I’ll pass it to ollama and it seems to do well with wittling it down for me, that’s about it.
I’ll say it has been marginally more useful integrated into my code editor, prompt driven for me has been useless output for too much effort, but it ambiently sitting there in code editor can be helpful.
I still can only get it to provide useful suggestions about 15-20% of the time for like two lines, maybe a nice error message, but the failure rate is less obnoxious if you didn’t spend extra effort to ask for it and just ignore and keep going. Getting a feel for whether or not the LLM is likely to have something in the completion worth trying to review is a part of it based on what you’ve typed helps. Notably if you are some keystrokes into a very boilerplate process you might be more optimistic, or if you are about to provide a text string as a human error message, decent chance it wrote that for you well enough.
Still I’m more annoyed and not sure that it’s worth being annoyed, but I could buy that shaving typing out a couple lines 15% of the time could be an objective boost that outweighs the burden of futzing with the high error rate.
I think your missing something big here.
A while back when one of the popular models came out it blew all the others away in benchmark tests. That got my coworker and boss super excited, we started coming up with different ways AI can help us. Thankfully as I pointed out, our software is proprietary and super secret, and all it took was a couple Google searches to find out a lot of those companies leak data like crazy and AI will just tell other people your secrets if they ask in the right way. So we needed to run our LLMs locally, but for that we’d need some beefy specs. I did the research, wrote a neovim and sublime plugin to integrate our local LLMs in a ‘copilot’ kind of way. My boss ordered my coworker and I whatever the new macbooks are with 128gb of memory to fit our lovely AI models, bought me a desktop tower and a few GPUs. Then I went on a 3 month paternity leave, came back and have heard nothing about AI since aside that my coworker switched to vs code to try and use continue but got frustrated, switched back to sublime and doesn’t use AI anymore.
So yeah, AI got me 2 new maxed out spec machines and 2 weeks of fuck around time writing plugins that were not nearly as complex as everyone thought, just because AI was involved.
For real though, I’ve tried quite a few times but 9/10 it fails me and I end up spending more time messing around with prompts than it would’ve taken me to do the thing. Occasionally when I have a mile long error message or something super obscure I’ll pass it to ollama and it seems to do well with wittling it down for me, that’s about it.
I’ll say it has been marginally more useful integrated into my code editor, prompt driven for me has been useless output for too much effort, but it ambiently sitting there in code editor can be helpful.
I still can only get it to provide useful suggestions about 15-20% of the time for like two lines, maybe a nice error message, but the failure rate is less obnoxious if you didn’t spend extra effort to ask for it and just ignore and keep going. Getting a feel for whether or not the LLM is likely to have something in the completion worth trying to review is a part of it based on what you’ve typed helps. Notably if you are some keystrokes into a very boilerplate process you might be more optimistic, or if you are about to provide a text string as a human error message, decent chance it wrote that for you well enough.
Still I’m more annoyed and not sure that it’s worth being annoyed, but I could buy that shaving typing out a couple lines 15% of the time could be an objective boost that outweighs the burden of futzing with the high error rate.