AI Implementation Failures: What We Learned from 2024

My news feed is filled with “A Year in Review” of what happened in 2024 and the thing that stood out to me was 2024 was a bit of a mess for AI implementations.

From chat-bots giving illegal advice to fake content flooding our news and social media feeds (I’m pretty sure that I’m not the only ones who’ve seen the Pope wear a cool puffy jacket)

So how did we get here:

The rush to implement AI solutions was largely driven by market pressure and FOMO (Fear of Missing Out). Companies, desperate to stay competitive, rushed to deploy AI solutions without proper governance frameworks or security controls. Board rooms worldwide echoed with demands for “AI strategy,” often without understanding what that actually meant for their business.

This perfect storm was further fueled by the accessibility of AI tools and platforms. What used to require deep technical expertise became available through simple APIs and low-code interfaces. While this democratisation of AI is generally positive, it led to a “wild west” scenario where implementations often outpaced proper security and compliance considerations.

The result? Poor deployment, Terrible user experience and many half-baked AI solutions, security vulnerabilities, and trust issues.


Before You Start: The Boring (But Essential) Bits

Look, I get it – you want to jump straight into the exciting world of AI. But here’s the thing: you need to sort out your data house first. Think of it like baby-proofing your home. Your CISO and security team need to know exactly what data you’ve got, where it lives, and who’s allowed to play with it.

Get your Microsoft Purview DLP policies sorted, tag your sensitive stuff using Purview Information Protection, and make sure you’ve got the right security controls in place. Trust me, this boring bit will save you from some proper headaches later.


The Fix: Four Simple Actionable Steps

  1. Sort Out Your Governance
    • Get an AI committee going
    • Write clear policies on AI usage, Data Protection, etc
    • Set proper standards
    • Actually check if things work (please audit!)
  2. Lock Down Security
  3. Quality Control
    • Keep humans in the loop
    • Test, test, test
    • Watch those outputs (again please run audit checks)
    • Clean data = better results
  4. Smart Implementation
    • Start small, scale later (even on a controlled Copilot for Microsoft 365, pilot it first with a handful of trusted people)
    • Train your people properly, (end-user education is a must)
    • Listen to user feedback
    • Don’t rush it

2024 showed us that rushing in without proper planning is a recipe for disaster. Take your time, do it right, and maybe we won’t see your company in next year’s “AI Fails” list.

Other Sources:

From Novice to Ninja: a new CISOs guide to DLP

Congratulations, CISO! 🎉 Great job in landing your new role, where protecting sensitive data isn’t just a job—it’s a daily tightrope walk over a pit of cyber threats, compliance demands, and evolving technology.

Now that you’re at the steering wheel, your inbox is probably overflowing with security concerns, regulatory requirements, and a few “fun” audit emails. Don’t worry, you’re in good company. This guide is here to give you actionable steps to set up your Data Loss Prevention (DLP) strategy, ensuring you don’t just survive in this role—you thrive.

So, what does being a CISO mean? Well, you’re now the go-to person when sensitive data sneaks out, malicious insiders get a bit too curious, or someone clicks that suspicious link promising free money from an unknown relative in Timbuktu. No pressure, right? But here’s the deal: inaction is risk. Delaying or overlooking the core elements of a solid DLP strategy could lead to breaches that cost more than your next cybersecurity budget.

To make your journey smoother, I’ve prepared a handy worksheet that you can use right now to take action on your Data Loss Prevention strategy. These aren’t just checkboxes—these are critical steps to lock down your organization’s data and avoid waking up to a breach nightmare.

You can Download the worksheet below.

Here’s what you can expect see inside:

1. Classifying Data and Why It’s Important

Why it matters: Not all data is created equal. By classifying your data, you can prioritize resources and security measures where they’re needed most. Would you protect the company picnic plan with the same force as your customers’ financial information? (Spoiler: probably not!)

Example:

  • High-risk data: Customer credit card details, proprietary code, or confidential HR files—things you’d never want to see in the wrong hands.
  • Medium-risk data: Internal meeting notes, marketing strategies—sensitive, but not catastrophic if leaked.
  • Low-risk data: Public reports, customer FAQs—this is the stuff you’d share at a conference.

Take Action Today: Review your organization’s data and start tagging it by risk level. Ask yourself, “What would happen if this data got out?” and use that to guide your classification efforts

2. Why and How to Identify Sensitive Data

Why it matters: You can’t protect what you don’t know exists. Sensitive data is often hidden across different platforms—sometimes even in the most unexpected places (like a random email attachment or NTFS file shares). Identifying it is the first step to ensuring it stays secure.

Example:

  • Sensitive Data: Personally Identifiable Information (PII) like social security numbers or health records, intellectual property (IP), and anything that’s subject to regulations like GDPR or HIPAA.
  • Surprise Discovery: Finding a list of client emails attached to a forgotten project buried in a shared folder.

Take Action Today: Use a discovery tool or audit your data manually. Start with cloud storage, email servers, and shared folders. Look for data that could lead to a privacy violation or financial loss if exposed.

3. Developing a Data Handling Policy

Why it matters: A solid data handling policy is the foundation of your DLP strategy. Without clear rules in place, sensitive information can slip through the cracks, exposing your organization to unnecessary risk. Your data handling policy ensures everyone—from top execs to interns—understands the dos and don’ts of handling sensitive information.

Example:

  • Clear Guidelines: For high-risk data like financial information, the policy might mandate encryption during transfer and restricted access to authorized personnel only.
  • Real-Life Scenario: Imagine your marketing team accidentally sharing a file with customer details over an unsecured network. A proper data handling policy would prevent this by enforcing secure file transfer practices.

Take Action Today: Draft a policy that covers how different types of data (high, medium, low risk) should be handled. It should specify everything from encryption requirements to access control and data retention periods. Involve key stakeholders (Legal, IT, HR) to ensure all bases are covered.

Now that you know the key steps to securing your organization’s data, it’s time to plan it out, partner with your internal stakeholders, and take action. DLP isn’t a one-person job—it’s a team effort that involves collaboration across IT, Legal, HR, and beyond. The risks of inaction are far too high, so don’t wait until something goes wrong. Proactively implementing these best practices today will not only protect your data but also strengthen your leadership as a new CISO.