As Elizabeth Transier points out in Governance and the AI Telephone Problem: When Agents Answer to No One,

The question for business leaders is not whether to deploy AI agents - that ship has sailed. The question is whether we will require a governance framework that preserves accountability within our businesses, or whether we'll wait until an incident forces our hand.

If we were to write a business case to support employing and retaining skilled business analysis professionals, the alarming level of organizational liability created by AI agents would be the obvious answer to the question, What issue are we solving?

Leveraging BAs to strengthen governance over agentic AI helps to protect the organization from AI-related risks such as incorrect automated decision, regulatory non-compliance, bias and fairness issues, and much more. In her new article for Modern Analyst, Adriana Beal talks about why even when agentic AI removes the need for manual analysis, strong business analysts will remain a critical asset to any organization trying to transition from a current state to a sustainable, better target state.

You can read it here:

Why Business Analysts Are Still Essential to Solving IT Challenges

P.S. As LLMs continue to excel, and in many cases outpace humans in many business analysis tasks, business leaders need to be constantly reminder not to “anthropomorphize” LLMs. At the same time that AI agents can now write and deploy code and solve complex math equations better than most people, they continue to struggle with real-world understanding.

AI providers like to imply that mistakes like this (missing obvious real-world context about the point of going to a car wash) are an issue of the free, less powerful models. However, users of paid versions haven’t stopped experiencing similar issues. Although AI is increasingly taking over tasks once reserved for human expertise, the need for vigilant, critical human oversight to ensure successful change has never been greater.

It’s not just in professional development that we tend to waste time taking a circuitous route when we could have taken a more direct path instead. It happens all the time in our work and life decisions, from which software stack or business model to adopt, to whether to live in an apartment or a house.

At its core, this issue arises from our strong tendency to pay more attention to information that confirms our beliefs while ignoring or downplaying facts that contradict our ideas. We don’t do this because we’re incompetent or lack problem solving skills; it’s what our brains naturally want to do. And if we don’t fight this human tendency, we may not only fail to see possible threats, but also close off our radar to better alternatives.

The best way to avoid this mistake is to always consider the possibility that our assumptions about our problem and our proposed solution are wrong.

Good questions to ask include:

What are we absolutely sure is true?
What has been treated as truth without being proven?

This is how we go back to first principles thinking (i.e., thinking about the basic facts of the matter). Then, rather than only looking for supporting evidence, we can start focusing on discomfirming evidence: information that contradicts or challenges our beliefs or hypotheses and prompt us to reconsider our viewpoints.

For example, consider the statement, “It will be much easier to launch my career in data science if I get this certification first.” Taking into account only positive evidence (examples of people who successfully became data scientists after getting the same certification) would give us a distorted picture of the validity of this claim. How many people can you find that got certified and never managed to get a job in the field? How many data scientists with the career of your dreams don’t have any certification?

When we approach a claim with falsification in mind, and consider both confirming and disconfirming evidence, we can develop a much more accurate picture of the validity of our ideas, and dramatically improve our decision-making.

-You may also like: Pick your feedback, pick your future

But what does it mean in practice? How can a person’s critical thinking skills make them more valuable (and potentially indispensable) for an employer using generative AI?

Critical thinking is the most powerful tool companies have against making wrong assumptions about AI that end up causing disruption of business operations and financial setbacks.

Take for example the case of the law firm whose lawyers were sanctioned for a motion that cited nonexistent cases earlier this year. There is abundant evidence that today’s generative AI tools aren’t able to reliably produce content free from errors or bogus citations (even those with an astonishing performance in bar exams). That means it’s entirely unreasonable to assume that their “cognitive abilities” can be trusted without strong supervision and oversight.

A critical thinker in charge of investigating the changes in process required to responsibly adopt AI in a law firm would be able to identify opportunities to taking advantage of the technology. Potential uses include saving time and improving outcomes by having AI generate legal document drafts and find pieces of evidence in large volumes of content. But a firm line would have been drawn at filing AI-supported briefs in court only after every fact and citation was checked and every legal reasoning and argumentation logic was reviewed.

Besides helping companies avoid the misuse of AI technology, critical thinkers also help them avoid the mistake others make at opposite end of the spectrum: assuming that AI tools are useless in real-word use cases because of its tendency to be “confidently wrong.”

Whether we like it or not, even with its “hallucinations” and unreliable outputs, general purpose AI has shown evidence of being a valuable tool to increase productivity and enhance the quality of work outputs.

Properly deployed, enterprise sanctioned AI tools (either based on a generalist model or a specialized solution) can help an organization develop sustainable advantages. For instance,

• Law case strategy. Use AI to brainstorm potential arguments, anticipate counter-points, and draft discovery plans, treating it as tool to assist and enhance, not replace, one’s professional judgment and due diligence.

• Training materials. Use AI to generate material to train employees in complex manufacturing processes. Rather than directly using the AI output in training artifacts, treat it as a draft to be reviewed by at least two knowledgeable employees before the final version is released.

• Software localization. Use AI to produce translations and culturally appropriate images and gestures to make a software application feel native to local users. Put knowledgeable professionals (who wouldn’t necessarily be good at the translation task but can be reliable reviewers) in charge of proofreading and fixing localization bugs.

In common in all these scenarios is the human supervision and oversight layer. (It’s possible that at some point we’ll have enough evidence that an independent AI tool can successfully take this task over as well, but until then, human guidance is the only acceptable path.)

Some may challenge the benefits of adopting costly technology that can only work well under strict direction and guidance of humans. But again, the answer here is to use your critical thinking to question assumptions, recognize your bias, and consider all the evidence before forming any conclusions.

For instance, I’ve been involved in multiple software projects requiring localization. In many of them, launches were severely delayed because we struggled to find culturally-aware translators in various languages. In one case, because I was born in Brazil, I was in charge of supervising the localization work for Brazilian Portuguese.

We managed to find one great translator, but the other hires were performing badly. In that project I had to spend a lot of time fixing the mistakes of the less competent translators, which caused other tasks to suffer. If I was doing this work now, I’d definitely find the budget to introduce an enterprise AI solution to help expedite the process. While it wouldn’t be cheap, nor yield perfect translations, neither was the effort to constantly recruiting and training more translators and, more importantly, the opportunity costs of a delayed time-to-market. Our localized launches, and path to revenue growth, would have been greatly expedited. (Speaking as someone who has done the job, manually reviewing and improving mediocre translations created by either humans or AI is way faster and scalable than producing them.)

Critical thinking requires us to consider evidence and avoid letting emotions cloud our judgement. If you foster this mindset, you’ll develop a superior ability to evaluate arguments pro and con the use of generative AI, and will be well-positioned to find suitable workarounds for the limitations of the technology. For anyone interested in becoming an irreplaceable linchpin in the age of AI, the disposition to think critically when others might not, is still your best bet.

Many BAs are no longer writing the steps in every use case, or drawing every process map. Instead, they’re focusing primarily on defining tasks or questions for an AI assistant, providing context to anchor the model's response, and adjusting the wording and re-testing when an output doesn’t meet expectations.

Some analysts I know are being proactive. They’ve voluntarily started using ChatGPT or perplexity.ai to expedite tasks such as getting ready for requirements workshops, documenting workflows, or summarizing findings about business problems.

A second group is trying to follow a request by their organization to start demonstrating use of a tool such as ChatGPT or Microsoft Copilot in their routine work, as part of an initiative to increase productivity.

In both circumstances, managing risks and ensuring a positive outcome is no easy feat.

Let’s look at a couple of examples:

Here, the main problem is clear: if the AI assistant wasn’t fed with input from users explaining their pain points, chances are that its output had little to do with the real challenges the team was facing.

As we all know, process improvement is a deeply collaborative activity. AI cannot replace the value of real conversations to uncover hidden assumptions and build consensus. The lack of context, combined with the “people pleasing” tendencies of the majority of large language models, is likely to result in well-written recommendations that sound plausible but completely overlook the real business need.

This second scenario highlights the risk of inadvertently exposing sensitive information to a 3rd-party provider. The manager making this recommendation is probably unaware that, due to the less robust data controls of ChatGPT’s free version, business content shared with the tool may be used to train OpenAI models even if the user has opted out of training. Of great concern here is the potential of trade secrets or personally identifiable information (PII) being accidentally added to the provider’s training data—and later exposed in outputs for different users.

The solution, however, can’t be the same as the one adopted by Samsung in 2023: banning the use of ChatGPT among employees after sensitive internal source code was uploaded to the tool.

Rather than simply creating policies that prohibit the use of AI (both difficult to enforce and an impediment to leveraging the technology to improve business operations), organizations must focus on ensuring they have the right frameworks to enable its effective and secure use.

Ultimately, AI assistants can enable a wide range of benefits capable of revolutionizing business operations. To help materialize such gains, business analysts will have to stay informed about things like best practices for prompting engineering, memory management, and AI risk management.

The good news, for talented business analysts who still feel a bit intimidated by AI, is that the skill set they developed to excel at the role will remain be invaluable for the foreseeable future:

• Critical thinking to frame the problem for deep analysis, guide, create the right context knowing what to emphasize and what to omit, provide appropriate human oversight to protect the business against unexpected AI failures.

• Strong communication skills to efficiently and effectively delegate tasks, refine prompts, and create feedback loops that ensures reliable outcomes.

Skilled BAs can also become instrumental in helping organizations create a robust framework to ensure that:

• Anyone working with an AI assistant understands the limitations of large language models, how the quality of their prompts directly influences the relevance and accuracy of the outputs generated, and the risks of unintended data exposure.

• AI usage is properly monitored, sensitive information is only submitted to enterprise versions of AI assistants with strong built-in data protection features, and potential threats are met with quick responses to prevent data leaks.

During the progress of any complex project, change is inevitable. It’s the failure to make optimal decisions to manage these changes that leads to substantial risks.

Decision-making compass: a powerful lever for business analysts

AI chatbots can have a positive impact on the results we deliver—we just need to be smart about using them to our advantage rather than against us.

2024 Trends in Business Analysis, and How not to Be Replaced by ChatGPT in the Age of AI

From our sister newsletter, Data Points:

Three mistakes causing your "data-driven" strategies to fail to create breakthrough performance

Some housekeeping…

If you received this post by email, it’s because you subscribed to receive updates at BealProjects.com.

To continue receiving news from us, please mark this address as ‘not spam.’

You can always see everything visiting bealprojects.substack.com or using the Substack app.

Please tell a few friends if you feel like it.