Shadow AI at AEC firms: what staff tool use shows about the approved tools

Each personal tool in use points to a task where the firm's approved tools fall short, or where no tool was approved at all. A designer paying for an image generator shows that concept imagery has no approved tool. A marketing coordinator pasting past proposals into a personal ChatGPT account shows that proposal drafting has no workflow and no verified source folder.

The same use also carries risk. Client drawings, fee data or owner documents in a personal account may be kept or used for training, which can conflict with client contracts. A workflow that runs on one person's private setup also stops when that person leaves. A survey finds both the unmet tasks and the risky data, and the policy that follows can deal with each.

A survey takes about a week. Talk with 5 to 10 heavy users for about 30 minutes each, and send a short anonymous form to the rest of the firm. Tell people in advance that the survey is to find tools worth approving, and that nobody will be disciplined for what they report, because otherwise the answers leave out the useful parts.

Questions to ask

• Which AI tools did you use last week, and for which tasks?
• What did you put into the tool: public material, firm documents, or client project files?
• Is the account personal, paid by you, or provided by the firm?
• About how long did the task take with the tool, and how long without it?
• Which approved tool did you try first, and where did it fall short?

Ask about recent tasks rather than general habits, because people describe what they did last Tuesday more accurately than what they usually do. Record each answer as one row: tool, task, data used, account type and time estimate. Twenty to forty rows is enough to see which tasks come up most often.

For the tasks that come up most, test before deciding. Pick one real job people do every week, such as an RFI response, a code check or a proposal section. Run the approved default next to one or two alternatives on the same files, with the people who do that job, for one week. Record the time taken, the rework, and whether a senior person would accept the output. Leadership and IT then decide to expand, keep or stop each tool on those numbers.

We score tools on six criteria: what data the tool can reach, the effort to adopt it, how hard it is to leave, output quality on your own samples, how it fits the systems you already use, and cost compared with subscriptions you already pay for. A test like this also gives staff a fair hearing: when someone says a tool cannot do their job, the hours on the firm's own drawings and documents settle it.

Licenses handed out without a named task are one of the stall points in why AI implementation stalls.

Sources: Microsoft and LinkedIn, 2024 Work Trend Index, May 2024, a survey of 31,000 people in 31 countries. The training figure is from LinkedIn data reported with the index.