Building quick and dirty applications is getting easier and easier. But building reliable and mature ones is as difficult as ever. If not more given the increasing expectations of clients after seeing all the flashy demos going around.
One example: there are now tens of “English to SQL” demos out there, mostly just asking OpenAI for the translation. Some go further and provide the list of column names to try to force the #LLM to generate a valid SQL query.
And they are quick to build, easy to use and do work, some times.
But when “sometimes” is not enough, they are completely useless. There is no smooth transition path from an LLM-based component that occasionally works to a fully deterministic and reliable equivalent component that does not imply a full reimplementation.
At least not yet.
