The Future of Engineering is AIOps

Author’s note: This piece was originally written for Musings on AI. Go check them out!

The job of software engineering as we know it in 2023 will look completely different in 5 years.

No, I’m not saying that software engineering is dying. Or that code is dying. Or that developers will be replaced by AI. ¹

Instead, I believe that software engineering is on a path to be commoditized, simplified, improved for the better by working hand in hand with generative AI tooling.

This isn’t specific to Data Engineering. Or Analytics work. Or even data modeling. If you happen to touch any code, this shift in mindset is going to overtake the work you do.

What is AIOps?

It seems like every business function now ends in Ops. RevOps. BizOps. DevOps. DataOps. So why not keep adding to the list with AIOps?

AIOps is the concept of working side-by-side alongside AI to develop solutions with code to improve business workflows. It is NOT the process of developing AI tooling itself.

Instead of focusing on writing code from scratch in an IDE to build out sustainable applications, the engineering focus for AIOps is shifted towards problem solving and the effective natural language articulation of a solution. The goal of AIOps is to create workflows that solve problems through a series of chained steps where the end result of each step is a working code block or solution.

The process of AIOps closely mirrors that of the typical software development lifecycle:

• Business problems are broken down into multiple steps that need to be accomplished.
• Each step will need to have three components:
  • The input file or data
  • The requirements, provided in natural language, to achieve the desired goal
  • The output result, generated by code that accomplishes the requirements
• The output results will need to be continually evaluated until it is verified to be accurate.
• Once verified, the resulting code will need to be deployed into a series of development, staging, and production environments.

Many of the outputs of AIOps will effectively be single-task microservices. These can be independently run and called by different applications or business users.

The best part is that this work can be performed with minimal knowledge of code itself. If you can define requirements, verify the output, and deploy the final results in plain language, you can work in AIOps. Stated another way, AIOps will not be restricted to engineers. It can be performed by “citizen developers”, those who know enough coding principles to be dangerous, and business users that are deep experts in their domain with ideas of solutions that will drive value. This will greatly open up the amount of problems that can be easily and quickly addressed by software.

With the basic definitions out of the way, let’s dive into how I came to the conclusion that AIOps is the future of Engineering, my key learnings from implementing AIOps for over a month, and what breakthrough ideas I believe could be spurred from this shift in work.

Exploring the World of AIOps

In order to come to this conclusion, I had to put my theory to the test. That’s why I spent 30 days exclusively coding with ChatGPT. I needed to immerse myself in the idea to see if it was all hype or if there were any merits to a change in the way we develop code.

What I learned from this experience ultimately shaped my beliefs on what AIOps currently is and what it could be in the future.

Breakthrough Ideas

After a month of testing the limits with a new technology under my belt, I started to regulate my engineering process which cemented the idea of AIOps being the future of work in this field.

At the same time, I was able to formulate a few ideas that seemed like they would go hand-in-hand with this new way of development.

App Store for Conversations

The interesting part about using ChatGPT’s Code Interpreter functionality is that you can share the resulting conversation with any other person. When they click on the URL, they can see the entire conversation and the generated code (although they can’t download any files that were generated).

At the bottom, you used to have an option to “Continue the conversation”. This allowed you to provide your own files and have the previously generated code applied to those. Unfortunately, OpenAI seems to have taken away this functionality within the last month or so.

See it for yourself: https://chat.openai.com/share/34177a88-b642-4bf7-a6e1-1d3295b08b54

What this revealed to me was the ability for companies to make an “app store” of conversation links. It’s probably not necessary for your very basic utility scripts, but if your business has a proprietary way of doing things, it could prove to be very powerful.

For example, maybe you use a service that allows business teams to provide a CSV to make bulk edits. You could have someone on your team use Code Interpreter to develop a “Bulk Edit CSV Validator” to check and make sure that the CSV is error-free and meets standards. Now, instead of building out an entire application UI to accomplish this functionality, you can provide an internal URL to the conversation. A business user could continue the conversation, upload their file, and get the desired results quickly.

With this mindset shift, you can create usable applications in minutes without ever writing code yourself, all while guaranteeing the accuracy of the “app” by simply verifying the output.

I see this as being quite different from the planned “Custom GPTs” which only allow you to provide source files and custom instructions with the hope that a job will be done consistently every time. From my own testing, custom GPTs are the equivalent of putting lipstick on a pig. It’s just a slightly modified experience of ChatGPT with too much variability to be useful for teams that need consistent, verifiable outputs. However, this could change as the product improves.

Conversations as Code

From the learnings I shared above, we can see that the code is never in an immediately deployable state, but there are consistent steps that can be taken to make it deployable. Additionally, any code that’s generated can always be shared as a standalone, version controlled conversation through a link.

This made me realize that there might be a way to provide a conversation link and automatically have the executable code extracted in your workflow engine. The conversation is the code.

Through some digging, I discovered that the conversation link itself actually provides a JSON representation of the entire conversation, including the code snippets separated out. Using the link I provided above, if you open up the source code, at the bottom you’ll find a <script> tag that contains all of this JSON.

Take a second and dig into the JSON to see all of the information that’s provided. There’s a lot of useful information that can be extracted!

This means that you should be able to outsource the development of modules of work to an AI model and easily import the results back into your execution engine of choice. ²

If this idea of “conversations as code” starts to see some movement, the development community might be able to enforce standards on the JSON output of all conversational AI models (similar to what we see for event tracking services like Segment, Rudderstack, etc.). This standardization would make it easier to switch between conversational models while still gaining the unified ability to deploy the generated code in your tool of choice.

Right now, the approach of “conversations as code” seems better suited for ad-hoc initiatives and one off scripts, but that’s not the only application. The same logic could be applied to the previous idea of an “app store of conversations”. Now you can have plug-and-play conversation modules from an app store that can be used as repeatable templates throughout a workflow engine. That starts to become an incredibly powerful proposition for teams who want to get work done quickly.

Conversational Workflows

While these ideas are great, they’re also very piecemeal. They allow you to solve only one small problem at a time. But what if you need to string them together to build a more complex workflow for your organization?

I think the next stage is going to be having a tool that allows you to build each input/output step conversationally. You provide an input and all of the requirements you know you have. You validate that the output looks correct. If not, you continue the conversation until it is. You then connect the output as the input to the next step and repeat the cycle until you’re done. ³

Each step of the workflow would be completely modular with code under the hood, represented by a conversation in the frontend. Any step could be modified by simply picking up the conversation where you left off.

Once you’ve connected all the steps together, you could click “run now” or set up a schedule to automate the workflow.

We already have tools like Zapier that allow you to describe the full workflow that you want to build with natural language to create a shell of the workflow. If we can turn building each step into an interactive conversation, we’re going to see a lot more people able to solve their own problems without even needing to know code.

Putting Ideas into Action

To test these breakthrough AIOps ideas out, I put together a proof of concept to see how easy deployment could be.

In order to see if I could import code from a conversation, I needed to break the problem down into individual steps:

• Collect a ChatGPT conversation URL from the user
• Download HTML from the provided URL
• Parse the HTML to extract and store the JSON
• Reformat the conversation into a markdown file that was formatted to look like a movie script
• Prompt the OpenAI API to read the markdown file and create a finalized executable Python Script and a Requirements file.
• Process the the Python Script and Requirements file and transform them into the necessary JSON/YAML/TOML configuration for my deployment platform of choice.

Over the course of 6 hours of development performed with ChatGPT, I finally had a working proof of concept. It was ugly. It had to be executed by stringing together multiple separate scripts one after the other. Despite this, it worked quite well and could handle most examples I threw at it. The idea was viable, but at this point, was it too early for anyone to even consider using it? ⁴

I saw this vision come to light when a customer of mine needed to run some custom transformation logic on data that was downloaded from Tableau. The file had to meet very specific standards before it could be sent to their clients and the team wasn’t able to write code themselves. To solve the problem, they were able to upload a sample file to ChatGPT, provide the requirements, and create a script that gave them the desired output. They then shared the link with our team and we helped integrate the code as a “middle transformation step” between downloading and uploading the data. It was manual work, but proof that the concept is sound.

What the Future Holds

Ultimately, I’m convinced that this is going to be the new model for engineers. Not because someone keeps nudging me about it. Not because I read about it online. Not because I’m invested in hyping AI solutions.

It’s because I actually experienced it myself and realized that the engineering process was much better than what we currently have. It’s quicker. It’s simpler. And most of all, it’s enjoyable.

As the generative AI models continue to improve, the future of engineering looks bright. It may involve less engineers, but it will surely involve less grunt work and more people being empowered to bring their solutions to life. That’s a win in my book.