Human-in-the-Loop as a Leadership Practice: A Framework for Better AI Workflows

A shift is underway that has implications for leaders developing AI-assisted workflows. In recent months, even some technology leaders have begun revisiting an idea that once seemed unfashionable: the enduring value of the humanities. As AI becomes more capable, qualities often associated with disciplines like psychology, philosophy, and literature are being reframed as professional advantages rather than academic luxuries.

In a recent New York Times opinion piece, columnist Maureen Dowd explored the renewed interest among AI technologists in liberal arts education. In the piece, sources suggest that a deeper understanding of human behavior, ethics, history, and enduring narrative themes might help younger professionals gain an edge in an AI-rich workplace. The notion is striking: the more sophisticated AI becomes, the more valuable distinctly human capabilities become.

The implications for leaders engaged in workflow design or redesign run deeper than the superficialities sometimes associated with Human-in-the-loop (HITL).

In this space, I have written about HITL as a safeguard for ensuring human judgment remains central to AI-assisted work. In a previous blog post, I argued that effective HITL requires cognitive friction—intentional pauses for questioning, verification, and reflection that are distinct from mere review and approval.

The challenge for today’s leaders is twofold: First, to ensure meaningful human engagement in AI-assisted workflows; and second, to ensure that humans drive the continuous improvement of those workflows.

That is where HITL as a leadership practice comes in.

In this blog post, I offer a practical HITL leadership model—a repeatable process for keeping human engagement intentional, your team’s relationship with AI collaborative, and AI-assisted workflows continuously improving.

What HITL Means to Leadership

Before leaders can use HITL well, it helps to define what it is and what it is not.

Human-in-the-loop (HITL) is a collaborative workflow in which AI assists, but human judgment remains “active and continuous” in guiding decisions and “mitigating risks.” (See “Human-In-The-Loop” What, How and Why” by Cyril Maréchal.)  It is distinct from human-over-the-loop, in which a human acts only as a supervisor. In other words, HITL is not simply a final review step. It is a design principle for preserving context, ethics, experience, and strategic control throughout the workflow.

The goal of HITL is to allow AI-assisted workflows “to achieve the efficiency of automation without sacrificing the precision, nuance and ethical reasoning of human oversight,” explains Cole Stryker for IBM Think. It allows humans to course-correct and helps provide an audit trail for decisions.

These distinctions matter because AI can generate speed and scale, but it cannot reliably substitute for human interpretation in complex or high-stakes situations. I have discussed the potential weaknesses of AI outputs in previous blog posts. (See “Safeguarding Content Quality Against AI ‘Slop’” and “Leveling an Editorial Eye on AI.”)

Leadership research increasingly describes HITL as part of a larger workflow redesign process, one that includes “adaptive layers that connect humans and AI in continuous feedback loops.” These new “systems of work are designed for change” by allowing “human judgment and machine insight to interact in real time,” explains Keith E. Ferrazzi and Wendy Smith, reporting recently for the World Economic Forum.

But workflow and process design are not so easy.

The 5-D HITL Framework as Leadership Practice

To make HITL intentional and practical, I use a 5-D framework that helps leaders design workflows with built-in human oversight from the start.

The framework avoids two pitfalls:

• Merely adding AI on top of an existing process
• Merely adding a human checkpoint that is devoid of cognitive friction

Traditional HITL models often focus on preventing AI mistakes. That remains essential. But as organizations mature, HITL must do more than control risk. It must also create learning loops that improve how humans and AI work together. To that end, my framework offers a repeatable method for shaping AI-assisted work from the beginning, so managers and team leaders can clarify the task, guide the AI, evaluate the result, and improve the process over time.

To design it, I drew inspiration from several sources, including AI governance principles, continuous improvement models, and professional practices in project management and communication. I also reviewed documentation by organizations promoting responsible AI use, including work by the National Institute of Standards and Technology (NIST), UNESCO, and policymakers developing the European Union AI Act.

The model is intentionally cyclical. Rather than treating AI use as a one-time interaction, it treats AI-assisted work as an evolving process—one that improves through human judgment, careful documentation, and thoughtful refinement. Unlike a checklist that ends when a deliverable is approved, the 5-D cycle creates a feedback mechanism. Each completed workflow should improve future prompts, policies, practices, and human-AI collaboration.

Although individuals can apply the 5-D framework to their own AI use, its greater value emerges when leaders use it to design repeatable practices across teams.

Why Leaders Must Own Workflow Evolution

Research by MIT Media Lab and others has shown that most (95% of) business investments in GenAI pilots do not deliver value or move to production. Harvard Business Review calls this the “AI experimentation trap.” 

Because leaders own the outcome of a workflow, they must, by default, also own its successful implementation—and evolution. In an AI-aware workflow redesign, leaders must decide where AI can accelerate work, where human judgment must remain in control, and how frequently teams should evaluate AI use.

As such, the implementation of an AI-assisted workflow should not be static, argue Ferrazzi and Smith. Not one and done. “Rather than treating AI as a project with a clear endpoint, organizations [should] treat it as a capability that develops over time.”

That means HITL is not just a safeguard; it is also a means to an end.

Initial HITL process design “considers how people think, decide, and create” and then “operationalizes” roles and workflows to leverage the strengths of both humans (judgment, creativity, and accountability) and AI (speed, scale, and pattern recognition), according to slalom.com. Leaders who oversee both people and processes strive to improve productivity and efficiency through this balance.

Then, during a final “loop” of HITL, workflow and roles evolve based on lessons learned. Approvals, escalation records, change requests, and audit logs become part of this continuous improvement feedback loop for the process. This feedback loop complements the corrections and other feedback that humans give to the AI tool, thereby generating new training data and improving the tool.

But this use of HITL must be intentional. Let me show you how.

The Framework 1-3: Define, Delegate, Distill

The first three steps focus on the front end of workflow design: Define, Delegate, Distill. They support the idea that work teams should not simply ask AI to “help”; they should design the task so the output can be meaningfully evaluated.

1. Define: Frame the Problem Before the Tool Does

Every successful AI interaction begins before the first prompt.

The Define stage focuses on clarifying the task, desired outcome, and boundaries before AI enters the process. Leaders often underestimate how important this stage is. Yet unclear goals frequently produce unclear outputs.

Before designing a workflow that delegates a task to AI, ask:

• What is the actual goal of this task?
• Who is the audience?
• What level of detail is appropriate?
• What risks or sensitivities should I consider?
• Is AI appropriate for this task at all?

This final question matters more than many organizations acknowledge.

Not every activity belongs in an AI-assisted workflow. Tasks involving confidential information, sensitive stakeholder relationships, legal interpretation, or ethical ambiguity may require more human involvement—or may be inappropriate for AI altogether.

2. Delegate: Set Expectations and Guardrails

Once the task is defined and carefully bounded, the next step is to Delegate the task to the tool.

This stage focuses on setting AI up to succeed by requiring a crafted prompt that provides clear instructions, useful inputs, and meaningful constraints.

Prompt libraries help ensure consistency with this step. Repeatable prompt formulas are also useful. Additionally, leaders should encourage team members to provide background, expectations, examples, and parameters with their prompts.

Checklists for successful prompt creation should also include questions such as:

• What information does the tool need?
• What format should the output follow?
• What should be included—or excluded?
• What organizational or governance constraints apply?

This step is also where governance enters workflow design. Workflows should always defer to policies that define expectations around:

• Acceptable AI tools
• Data privacy and protection
• Prompt libraries and templates (use and maintenance)
• Disclosure practices
• Security requirements

Reminder: Public AI systems should not be treated casually. Sensitive, proprietary, or personally identifiable information should never be entered into tools that do not meet organizational security requirements.

3. Distill: Refine, Verify, and Add Human Context

If there is one stage where human value becomes unmistakable, it is the Distill stage.

AI can generate a first draft. It can identify trends and summarize data. But it cannot fully understand your organization’s culture, stakeholder dynamics, or institutional history. That remains human work.

This stage is where a workflow requires a human to interpret, refine, challenge, and improve AI output before it becomes action. When designing HITL workflows, I recommend adding in cognitive friction pauses that focus on the 5Qs:

The 5Qs—the five questions or five quality checkpoints—require a human to examine the AI output. In general, they ask, is it:

• Relevant?
• Accurate?
• Complete?
• Aligned?
• Fair?

These questions help team members move beyond superficial review toward meaningful evaluation, ensuring that the work effort fits the audience, task goal, and context. For example:

• A status report may appear polished but omit emerging risks.
• A stakeholder summary may be technically accurate while missing important political realities.
• A recommendation may appear efficient but conflict with organizational values or long-term goals.

Effective HITL requires moments of deliberate questioning—brief pauses that prevent us from accepting plausible output too quickly.

The Framework 4-5: Decide + Disclose, then Develop

The final two steps add governance and continuous improvement. They help us remember that HITL is not only about control but also about transparency and learning.

4. Decide + Disclose: Own the Outcome

The fourth stage reinforces a principle that humans cannot delegate: accountability

AI may assist analysis, drafting, or recommendation-making. But ultimately, people remain responsible for decisions and outcomes.

At this stage, the humans engaged in the workflow must decide:

• What moves forward,
• What needs refinement,
• What risks remain acceptable
• How outcomes should be communicated

Before finalizing work, each AI user should ask:

• Have I included everything my audience needs?
• Have organizational requirements been met?
• Could I defend this decision later?

That last question is particularly important. Increasingly, organizations and regulators are emphasizing traceability—the ability to explain how decisions were made and what role AI played in the process.

Additionally, some organizations now require disclosure statements for AI outputs that are included in formal documentation or are public-facing. Read more about disclosure in my blog post “Why Your Company Needs a GenAI Policy for Content Contributors.”

Even when formal disclosure is not required, transparency matters. Remember, stakeholders are more likely to trust AI-assisted work when they know humans remain actively involved, validate outputs, and maintain accountability.

5. Develop: Improve the Workflow Over Time

The final stage—Develop—is what transforms HITL from collaboration into continuous improvement.

We are tempted at times to treat AI interactions as isolated events:  Prompt → Output → Finished. But effective AI-assisted workflows improve over time.

Every HITL workflow should include a review step. Adding this step enables teams and leadership to regularly ask:

• What worked well?
• What could improve next time?
• Were better prompts available?
• Did we miss important context?
• What lessons should the team share?

With this step included in a workflow redesign, teams, over time, can develop:

• Stronger prompt libraries
• Clearer governance practices
• Improved review criteria
• Shared lessons learned
• Better judgment about when—and when not—to use AI

This final step is where leaders move their organizations from AI adoption to AI maturity. The measure of success is not simply whether teams use AI, but whether they learn how to use it responsibly, consistently, and effectively.

The Leader’s Role(s) in HITL

HITL is a leadership responsibility. Leaders set the boundaries for acceptable AI use—including governance and policy—define the conditions for review, and establish how teams learn from the results. (I wrote about AI policy in last month’s blog post.)

Additionally, HITL-based workflows reinforce the value of human-centered leadership. The Center for Creative Leadership (CCL) reminds us that leadership is a “social process” that occurs “when direction is shared, work is aligned, and people are committed to collective success.” Leaders facilitate the trust, relationships, and shared understanding that drive those conditions.

Thus, human leaders in an AI-rich workplace have several roles:

• Advocates – Championing the development of AI guardrails, including governance, policy, data protection and privacy, security, and tool-specific safeguards
• Mentors – Guiding staff by providing training opportunities, encouraging the honest exchange of ideas, and modeling responsible AI use
• Change Managers – Initiating pilots, recognizing challenges, addressing issues, setting up change logs and similar lesson captures, and moving the group from experimentation to implementation
• Sense-makers – Demystifying AI, clarifying its purpose for the organization, connecting people through shared understanding, fostering the conditions for alignment and commitment, and shaping the broader narrative
• Conductors – Integrating human and machine capabilities into a coherent whole with thoughtful choices, coordination, and alignment across a full spectrum of workplace needs and tasks
• Guardians – Protecting the human core of the organization by upholding accountability and ethical decision-making, protecting psychological safety, and reinforcing human judgment

The last three points stem from CCL’s work on AI convergence and leadership, which likens leaders to musical conductors who “must orchestrate meaning” in an AI-integrated work environment. In that sense, leaders are not simply adopting tools; they are shaping the conditions under which AI-assisted workflows become useful, trusted, and maintainable.

Ultimately, leaders shape not only AI workflows but also the culture surrounding AI use—what teams question, what they value, and what they choose not to automate.

The Best of Both Worlds

The renewed interest in psychology, ethics, history, and narrative reminds us of something important: successful leadership in an AI-rich workplace depends increasingly on distinctly human capabilities.

Organizations will continue to benefit from the automation, speed, and synthesis capabilities offered by AI. But the best AI-assisted workflows are not fully automated; they are intentionally designed around human responsibility.

Leaders do not just supervise AI after the fact — they define the conditions under which AI can enhance, not replace, the human experience in the workplace. That is the core value of HITL as a leadership practice. It keeps human judgment central while giving teams a repeatable way to use AI more effectively.

Declaration of Generative AI and AI-assisted technologies in the writing process: The author used ChatGPT and Perplexity to assist with the research, outlining, titling, and drafting some sentences of this blog post.