Hi everyone. I’m back. I’ve taken a two month hiatus from my Substack to make more public noise about The Skill Code. I’ve been on 21 podcasts with 16 more on the books. Here’s a recent favorite: Second City’s “Getting to Yes, And” podcast - yes, the improv comedy shop that spawned Tina Fey, Stephen Colbert, Jane Lynch, and Bill Murray. And I have published 24 excerpts, summaries, and OpEds in venues such as the Wall Street Journal, Harvard Business Review, BigThink, Forbes, the Next Big Idea Club, and the Financial Times. Who knows what will follow.

One thing is clear, though: the book has started wonderful conversations, has provoked new thinking, and has instilled a sense of urgency in a wide range of people. Managers and leaders and L&D types came out quickly with reactions. But also parents, uncles, aunts, policymakers, investors and even college students. I’ve been inspired and intrigued by the reviews that these have posted online. Folks are worried about AI. Or complacent. But I keep hearing one thing over and over again - from literally everyone - some version of: “I hadn’t thought about this problem!”

“We’ve started a war between technological productivity and human skill, and skill is losing.” (p139, The Skill Code)

I hear this from all quarters, most notably from those with quite significant power to do something about it: leaders, managers, and human resources professionals in organizations. I’m in a lot more contact with people in these roles now - people with serious experience, a healthy chunk of them running brand-name megafirms making big bets on generative AI in the flow of everyday work. “I have 20,000 Copilot licenses to use,” these people tell me. These people have known for a decade (at least) that we are facing a serious skills gap, and that the way organizations measure workforce readiness is ridiculously out of date, and that AI offers huge, quite uncertain promise as a tool to remedy all that. It’s refreshing to speak with professionals taking these issues so seriously. Yet so far, they have come to me saying the above. Though they might have heard a story or two, and many of them had intuition that something was amiss, they had not considered the possibility that their firms were deploying intelligent technologies in ways that compromised the richest source of skill development they had available: expert-novice collaboration.

Seeing below the surface, through conversation

None of this should be surprising. In fact for me it should serve as gratifying confirmation. I had to spend two and a half years in operating rooms in top hospitals, then another three spidering across over 30 other contexts and datasets to see this subtle negative consequence of standard approaches to automation via intelligent technology. Then write a book to clarify my own thinking and bring together the best available data. All my work, primary data, and careful scrutiny should make it perfectly clear the “war” between technologically enhanced productivity and human skill is not exactly obvious.

But hearing all this back from folks made me ask a new question: why?

That is to say: why isn’t this tradeoff between productivity and skill development obvious? I realized I didn’t have a good answer.

We’ve had theories of deskilling from automation since at least the late 80s. But they clearly weren’t top of mind, perhaps in part because they didn’t speak to collaboration across expertise levels. And it turns out the research here is equivocal - in some cases technology leads to deskilling, in other cases it seems to help. Steve Barley wrote a *fantastic* essay on this in the 1980s, basically suggesting that people who found deskilling had looked in places where you’d most expect it to happen.

My stock answer on podcasts - that I still think is not false - has been that no one really “owns” this problem. To begin with, everyone in an organization clearly “owns” productivity. Workers are personally responsible for the quality and pace of their output. Managers and leaders are responsible for the same for the people, functions, and firms they manage. And all parties contribute to and refine ways of tracking progress there. But who owns whether the proverbial saw stays sharp? That’s a maintenance and technical readiness function. Secondary concerns that play out over a longer time horizon that are hard to square with the white-hot concern of more, better, faster, cheaper.

Workforce readiness - a broader set of concerns - is typically owned by human resources, learning & development, and the like. But these folks are often treated (and sometimes consequently treat themselves) as second-class organizational citizens, have limited budget, and spend that budget and their professional attention on issues that revolve around formal training. Worst case they end up focused primarily on compliance training that has little to do with workers’ value-adding abilities, best (typical) case they end up focused on training for the “skills of the workforce to come” or similar.

My book hammers home the main issue there: most of our valuable skill comes through doing our jobs, not formal training. In fact, too much formal training hurts skill development. And we’re definitely doing too much, relative to our investment in the informal process.

None of this is wrong, I’ve realized. It’s just incomplete and misses the fundamental point. There’s something much deeper at work here, something that powerfully explains why the trillion-dollar threat evident in my book isn’t quite on anyone’s radar screen.

You get what you measure, and we’re missing a ruler

tl;dr? We don’t have a measure for the effects of simply doing a task on the skills of the doer, and that stops us from understanding effects of changing that task for the sake of improved productivity.

That’s right, the problem is that we’re missing a ruler.

The research on metrics and evaluation is very clear: until you create a metric to describe something, it doesn’t count, compared to things that are measured. When we invent a low-ish cost way of measuring something, we begin to see it as a legitimate thing to pay attention to, and to shape our decisions around it. Don Kieffer (former VP of operational excellence at Harley Davidson) taught me this when I saw him teaching some executives, and he asked them: what comes first, millimeter-scale manufacturing equipment or the tool that can measure millimeter-scale tolerances in that equipment? The tool, not the equipment. The minute builders, maintainers, leaders, and managers could see that their equipment wasn’t meshing well at that scale, they demanded equipment that could.

I’m talking to a lot of leaders in corporations these days. People who have bought 5, 10, or 20 thousand Copilot or ChatGPT licenses for their workers to use. Some don’t quite know where to deploy them, and are concerned about wasting the opportunity. Some have - a recent survey by UpWork showed 77% of employees feel like their workload has increased because of the technology. Some firms are much more confident because they have a refined sense of where to deploy, thanks to help from my colleagues and friends at Workhelix. And a few of them are, in a sense, past-masters at this already: some high tech firms are over a year and a half into AI deployments for their software engineers. But all of them are hoping for, gathering data on, and measuring one thing: productivity.

A few - a very rare few - have begun to sense that something’s amiss. That pushing so hard for productivity is getting tangible results (at best) but is making it much harder for junior workers to build skill and get ahead. I’m hearing this directly from leaders and managers as I talk with organizations these days. Here’s a recent *superb* essay by Steve Yegge on this topic from inside the belly of the beast (working at AI startup Cody). He flags his essay as “speculative.” He hasn’t read my book, and I doubt he’s aware of my research. If he were, I doubt he’d have put that disclaimer up there.

But none of these firms have data on this sacrifice of human potential on the altar of ai-enabled productivity, or even a clean way of thinking about the problem. The zeroth step, I’ve realized only just recently, is that the only way this issue is going to get noticed or systematically addressed is if the entire world gets a new ruler to measure it.

We’re building that ruler

Last fall, I began working with the fantastic Brandon Lepine (PhD student here at UCSB) to understand how we use generative AI to solve complex problems. If you’re a longer-time reader, you may remember my post that described how we challenged our Master’s students with almost no coding expertise to do data analytics in python and build software useful to a project manager with only genAI as a guide. They all succeeded, in spite of their fear and limited skill. In the background, however, Brandon and I were breaking down their quite complex problems, collecting their chat transcripts, interviewing them, and assessing their knowledge of coding. Why? We wanted to see how they solved problems differently, with genAI in their hands, and what the likely effects of their usage patterns were - on their productivity, sure, but also… their skill. What did they learn, or fail to learn, because they had handed off certain portions of their task to generative AI? Where did they gain in productivity?

We’ve gone a lot further in this direction since. Much more to come on this in a subsequent post, but the short story here is that we are working with some very exciting partners (maybe, next time, I’ll have their clearance to share) to develop a method for examining work that will allow fine-grained predictions about productivity and skill at the same time. That’s important - essential, really - for two reasons. The first is solely focused on productivity. The amazing fact is that while firms have some course measures of productivity changes as a result of genAI deployment, most don’t. They’re just handing out licenses, doing a bit of training, some experimentation, and mostly… hoping. Honestly. And for those that did have good measurement practices and good quality data, they don’t really know why they’re seeing the results they’re getting. They can make coarse distinctions. Useful, but not precise enough for great decision making. Our approach offers dramatic resolution improvement there.

But the more interesting, likely more valuable reason is that right now we think of productivity as what you can get as you automate and redesign work to suit - but not that it might come at the expense of human ability. Again, this shouldn’t be a big surprise, because the core message of my book is a surprise. Leaders, managers, HR and L&D professionals, and even workers usually agree hear that these things are interrelated, and they likewise agree with the finding that says we typically trade away skill development for productivity. But out there, on the ground, this isn’t a ready intuition. That means that as we shoot for increased productivity through genAI-driven automation, many of us will miss situations where insisting on automation patterns that *improve* human ability might actually help you get even more productivity than if you were measuring and intervening on productivity alone. We don’t have science that explains when and where these “both/and” opportunities will emerge, but my and others’ research makes it clear such outcomes are systematically achievable.

Juho Kim - a truly brilliant HCI researcher and collaborator of mine over the last four years - got involved in this effort recently, and he recently framed all this in terms many data-driven managers would relate to: joint optimization.

I talked a lot in the last two chapters of my book about the positive role technology can - and in my view, must - play in a skills future that is brighter than we ever could have achieved before. But if you read that closely, you’d see: I explored examples of how we could use technology to enable better, richer work and human collaboration - not how we could use AI to even *measure* the problem in the first place, so that people would be motivated to address it.

Now, we’re clear: until we have a ruler that measures (and therefore can predict) the interdependent effects of generative AI deployment on productivity and human ability, we’ll get dragged towards the former at the expense of the latter. This of course goes for many other kinds of automation via many other kinds of intelligent technologies, so the returns to this kind of new metric should unfold at the same scale as the problem.

As I said, stay tuned - Brandon, Juho, our collaborators and I will have results to share soon. And if you think your organization, division, or functional group is implementing generative AI in a way that subtly (or not so subtly) sacrifices human ability on the altar of potential productivity - drop me a line. You might just be able to join in on our early efforts to build just the ruler you need to find out, and to bend the arc of implementation towards a win in both categories.

Also, I should say that I’ve got… something else I’ll be announcing soon. It’s in the same “space” as the book, quite possibly a FAR bigger deal. That’s also been taking up a lot of my time.

But here’s what got me away from all that: a bunch of well intended, very common, dangerous advice out there about how to handle genAI in organizations. If you or your organization follow this guidance, things will not go well unless you’re lucky. Save those kinds of odds for Vegas - let’s review the science.

But first, to the problematic guidance.

Ignore, Ban, and All Hands

We’ve had three predictable approaches arise in the “how to do genAI in your organization” zeitgeist in the last year.

The first coherent organizational genAI strategy that emerged was ignoring it.

Ask around, and this is still mostly what you’ll find. Leaders at companies might have heard of AI, but they don’t know much about it other than what they overhear as their kids “cheat” on their homework. But - and I mean this seriously - that’s the beginning and the end of their exposure. And their decision making isn’t going to change. If you’re Marcus, the guy (aka research subject) I know who owns and runs a mom-and-pop third party order fulfillment company on the eastern seaboard, you saw a Super Bowl commercial about AI, and you kind of shrugged and went back to trying to get orders, manage costs, and keep your warehouses productive for your customers. It’s a brutal business, and the only time you adopt a disruptive new technology is… never. That’s right. Never. You just don’t. Your margins are razor thin, your customers are hyper-demanding, and your workforce is a dynamic problem. If someone can sell you some technology to solve a real problem, reliably, starting yesterday, you’ll buy it, but the case has to be crystal clear and even then they have to call three times.

More commonly, you identify a need (say a tripling of demand for hand sanitizer during COVID), you realize you can meet it with tools you have on hand (a nail polish filling machine), you find a few more of those on the secondary market… in Canada, and you send a crew up there in a rented truck to pick them up the next day. Digital tech like barcode scanning, warehouse management systems, timecard systems… these are the bleeding edge of what you deal with.

So you ignore AI. In fact, you won’t buy a damn thing with those letters attached, because it sounds unreliable. Artificial? Sounds lower quality. Intelligence? Well, you’re familiar enough with the human kind. Suffice it to say you’d rather rent a conveyor or a spreadsheet than a person, if you can.

The next organizational strategy that emerged was banning generative AI.

We got a vivid, front row seat on this one in education, and many of us still have it. Teachers and administrators quickly got wise to the fact that students could input their homework into Bing or ChatGPT and get B+/A- material out in seconds. And its range was dizzying - everything from creative writing to history, coding to complex word problems, book reports to resumes. They concluded - accurately, in many cases - that unrestricted use of genAI on homework would destroy the learning value in those assignments. Struggle, careful thought, mistakes, and useful feedback are the hallmarks of the learning process, and all of this could get shortcut by a few keystrokes. This looked like an existential educational threat, so…

Many schools banned the technology. Entirely. Part of why they moved so swiftly in this direction had to do with the illusion of enforcement - plagiarism and cheating detection software had gotten quite good for previous, specialized problems, and administrators and teachers convinced themselves that these would do just as well at detecting generative AI use. We knew within weeks that this was patently false, but the belief and the practice of detection (and subsequent punishment) continues. Some teachers and administrators probably know this and just tell their students that these technologies could sniff out generative AI output, hoping the fear of punishment will curb its use.

Regardless, a ban remains. And while it doesn’t get nearly as much press, many companies have chosen the same path. Do not use generative AI, the word came down. Cheating was less the concern there than security - each time you prompt an LLM, the company that made it gets that data, and all of your responses in the prompting chain. And if that prompt includes company proprietary data, source code, or other competitively sensitive information, well… you’ re giving it away, and you’re probably fired. At a banning company, anyway.

A secondary concern here is the now well-known “hallucination” that genAI engages in. These systems produce very convincing output, and some of it is utterly made up. Yet human users can’t tell, have a strong incentive to save time and boost quality, so they accept that output and pass it along to those who will rely on it in some way. Then… oops. We all now know the story of the lawyer who wrote and submitted a brief using ChatGPT and got found out. But what, managers wonder, about the silent majority of users in my company who will do the equivalent but not get noticed? That’s like a creeping quality plague throughout the firm, one that no one has an immediate incentive to notice or remediate. What to do? Ban it.

The third strategy is “All hands”. This has only just emerged in the last few months, and is largely a reaction to the first two (and of course the broadening realization that we really have invented an imperfect, powerful, general-purpose cognitive prosthesis). For many leaders, managers, and business owners, inaction or resistance is clearly not an option - the only question is about how to engage. It’s also clear to those folks that no matter what they do their employees (or students) will use genAI to do their jobs, whether it’s via their smartphones or their work computers. They also know it’s not cost effective to try to detect this. And maybe they’ve gotten around to listening to or reading Ethan Mollick - that Wharton professor who seems to know all about it earlier than everyone else. Maybe they got Co-Intelligence, his new, NYT-bestselling book, maybe they found OneUsefulThing, his substack, or maybe saw him in the Wall Street Journal, or whatever.

Ethan lit a fire under everyone’s chairs - do nothing and you will be left behind. This is evident right out of the gate with his four principles for dealing with genAI. The first and foremost - the one he put in pole position for emphasis - is “Always invite AI to the table.” Ethan tells us all we should be playing around with this stuff. As in everyone in your company. Right now. Because nobody quite knows what genAI is good for, and those organizations that figure it out quickly are going to win. Just like kids did in education, your employees are figuring out new hacks with a new general purpose technology, and you had better steer into the skid. Deeper into that same chapter in his book Ethan pulls back from this a bit, suggesting that leaders should get help from “their most advanced users” and offer incentives for innovation. That’s better, but it’s buried.

Whether they were influenced by Ethan or not, a new class of leader and manager has emerged - one hellbent on granting full, paid, open access to LLM-based chatbots to every employee in the company, and telling them all some variant of “we need to learn how to handle genAI together”. Announcements go out. Enterprise-level OpenAI subscriptions get bought. And the organization lurches towards the technology, maybe faster than it’s lurched towards anything in recent memory. Hard not to hear the words “ChatGPT” or “AI” in the halls, the bathrooms, or on videoconferences. In fact it’s hard not to interact with genAI in these contexts - many folks get genAI-generated summaries of their online meetings for instance, complete with action items and participation feedback, for example. In this organizational mode, genAI is the new electricity, and it’s always on.

Science says: none of these are good enough

First, let me acknowledge that all of these organizational approaches have significant strengths, and will probably save the day in isolated instances.

What? Ignoring and banning genAI might be ideal?

That’s exactly right. I wrote about this a few months ago in “AI is fast, Automation is slow - Can they meet?”, so if you want a deeper dive into the research on automation and new general purpose technologies in organizations, you can head over there. But, tl;dr: most of the time, going early and fast with new, disruptive technologies does not end well for organizations. A few succeed, but most will hurl themselves at the problem in the wrong ways, or at the wrong times, and they will spin their wheels, burn money, and harm people (sometimes literally). The thing is we have no research (that I can find) that shows the organizations that survived or even thrived because they did not hurl themselves at the tech, or that shows how frequently this is a winning strategy. The closest we come on that score is the research on “fast following” - in other words waiting for someone else to go first, learning from them, and going second. But the term “fast” there should be a dead giveaway - there’s no “slow following” research. We instead study successful adaptation, showing that it takes a while, proceeds through experimentation and dramatic investment, and so on. That research bias reverberates through business-school-style academic advice on what to do - and vigorous action is the core recommendation. Ignoring new tech or even banning it aren’t in the playbook, even though those are likely better bets for many, at least for a while.

But that’s not to say that “all hands” is all wrong. It will save the day for some organizations, too. Especially when the cost and pace of experimentation is low and the technology is new, a huge swathe of research on technology and organizing tells us that if management adopts a new technology, rank-and-file employees will rapidly find ways of slotting that tech into their work, find new failure modes and risks, and incrementally adapt things to suit. The process is often messy but can get good results, providing employees at all levels with useful intel to guide decision making. And we have great research that shows that firms that invest in advanced automation like robotics tend to outcompete firms that don’t. Firms that invest hire more people because they are more productive, can grow as a result, and take customers from other firms. It’s those other firms that shed jobs.

But any savvy manager - or employee - needs to face facts: all three of these approaches are hamfisted. You wouldn’t try to eat a four-course meal with just one utensil, and if you take any of those three approaches above, you’re doing the organizational equivalent with genAI: ignoring a mountain of superb science on innovation, technology and organizing and organizational change. Put together they offer a far more nuanced, but clear, and tested organizational guidance that will greatly increase your odds of success with genAI.

Science-backed genAI implementation, ftw

The first core insight is that a lot of the tactics that help an organization adapt to disruptive new technologies get implemented outside that organization. That’s an invisible dimension in the three tactics above - they’re all done on the inside. So, lesson number one, if you don’t want to hobble your efforts: you’re we have to watch for tactics that apply on the inside and the outside. The other critical dimension that’s muddled together above is the individual/collective one. Some tactics apply to groups of people, while others are much more atomic, having to do with individual humans at work. Lesson number two is be sure you’re choosing the right activity level for the tactic you’re choosing, and don’t leave good options off the table.

The second you parse the field this way, then take a look at the research, a host of targets and tactics pop out. I’m not going to provide an exhaustive list here - just a few in each category to get you thinking. The core message is: effective organizational adaptation to truly disruptive technologies is liable to require multiple approaches drawn from all four squares, and the savvy manager will go hunting through the research to find those that fit.

So here’s that 2x2:

A detailed breakdown of even this subsample of tactics - and their relationship to organizational adaptation to disruptive technology - would take way more space than you’d allow, and wouldn’t be all that fun. So I’m going to buzz the tops of the trees here, give you a few links to relevant research.

Collective, Inside

Skunk Works: When it’s time to pivot or try something new, special teams get formed within organizations, often isolated from normal operations, to innovate and develop new technologies rapidly. These teams operate under the principle of autonomy and minimal bureaucratic oversight, which fosters creativity and speed. Just got confirmation last week from an AI researcher in the mobile space that only just now (i.e., way too late) a big tech firm has set up a skunk works team and dramatically throttled their compute so they are forced to be efficient on mobile. That’s the idea. Reference: Skunk Works: A Personal Memoir of My Years at Lockheed by Ben R. Rich and Leo Janos (1996).

New Divisions: This is perhaps the most well known, best researched play: creating dedicated new divisions within the organization focused on leveraging disruptive technologies, allowing concentrated expertise and resources to adapt and innovate effectively. And yes, this means a new offering to the world, not a new internal function. This requires dramatic capital outlay, which means sacrifices in other areas. Reference: Christensen, C. M. (1997). The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail.

Structured Experimentation: This is perhaps the most important way to channel rank-and-file, individual experimentation with genAI at work so that productivity sustainably improves. This involves setting up controlled, systematic experiments to test hypotheses about how new technologies can be integrated and utilized within the company. Toyota is famous for a *fantastic* method here - called the A6 - that I helped Nelson Repenning teach to executive MBAs at MIT’s Sloan school. He and his collaborator Don Kieffer (former Harley exec) refined and improved it. If you’re looking for a “color by numbers” approach to getting organizational value from new technology, this would be it. Reference: The Most Underrated Skill in Management by Repenning, Kieffer, Astor (2017).

Learning & Development: It’s so subtle and utterly urgent that I’m hitting it again - last month I wrote “Learning and Development: The New #1 Organizational Function.” This goes way beyond formal training or even job aids or job rotation programs. In fact those are dangerous wastes of time in many cases right now. They’re all centrally-driven, and now the game is learning from your employees as much as helping them build capability and learning from each other. Humans don’t learn any differently than they used to, but we’re in the middle of a practically discontinuous change in the technicum, so L&D has a far broader mandate. Reference: Is Yours a Learning Organization? Garvin, Edmondson, Giino (2006).

Diverse Teams: Research is very clear on this - when discovery and invention are paramount, teams with strong cognitive diversity outperform those that are homogeneous. So beyond doing things liek cultivating psychological safety (obviously critical) Leveraging the diverse perspectives within the organization to foster innovative ideas and approaches is essential for adapting to and integrating new technologies. Reference: Why Diverse Teams Are Smarter by David Rock and Heidi Grant (2016).

Collective, Outside

Open Innovation Challenges: Yes, an organization can - and sometimes must - engage with collectives outside its borders to get the right new idea about how to approach its reality. Open innovation is one key way of doing this, and involves leveraging the knowledge and creativity of external contributors through collaborative partnerships or competitions to solve problems and develop new technologies or products. The classic example here is NASA getting a new solution to predicting solar flares from a retired radio systems engineer, but your organization could just as well run an open challenge about how to put genAI to use. Reference: Chesbrough, H. W. (2003). Open Innovation: The New Imperative for Creating and Profiting from Technology.

Acquiring Firms: Sometimes, it’s just flat out true that no matter how hard you try to adapt to a disruptive technology within your organization, you can “learn” faster by simply purchasing or merging with another firm that is getting there faster. They found the right mix of talent, tools, techniques, and other resources to get the job done - why reinvent the wheel, even if you could? For those outside this research tradition, this is a surprisingly common way for large firms to manage the challenge of changing with the times. Reference: Rules to Acquire By by Bruce Nolop (2007).

Individual, Outside

User Innovation: Eric Von Hippel (a former prof of mine) pioneered research on this incredible phenomenon - many times, it’s individual users out in the real world who invent at least the first version of new technologies, products, services, techniques or blends of these things. It was an EMT mountain biker who invented the camelbak, the now ubiquitous sack of water you wear on your back - he filled an IV bag with water, slapped a tube on it and tied it around his back for a race. Von Hippel has since made a playbook for finding these users and integrating their early discoveries back into the organization’s offerings. Reference: User Innovation (open MIT courseware, by Eric himself).

Customer Discovery: Somewhat akin to user innovation studies, this involves actively seeking insights from individual customers outside the organization to guide the development and refinement of technologies. Organizations do this all the time to learn what to build next. Why not do it to find out if your customers have found a killer genAI app? You might find one for yourself, and find a new problem you can solve for them in one fell swoop. Reference: The Four Steps to the Epiphany: Successful Strategies for Products that Win. Steve Blank, 2013

Individual, Inside

Hiring: Organizations hire all the time. Why not focus your hiring on talent that can help with your genAI problem? I don’t mean machine learning engineers here, I mean people who - in addition to having the qualifications for their job - know a thing or two more than most about how to put genAI to work. This fall, Alexis, an undergrad of mine, worked up a first-in-kind genAI portfolio to help potential employers see this more clearly. If folks follow her lead, this kind of hiring will get easier. Reference: The Hiring and Firing Question and Beyond by Barbara Mitchell.

Incentive Schemes: Ethan Mollick has rightly drawn attention here. Offer genAI bounties! Or status boosts (A title change, modified responsibilities, and/or increased visibility?) Managers know a lot about how to set up reward systems that motivate individuals - now just focus these on experimenting with and adopting genAI in ways that also help the organization. Doing incentives right is hard, but research is pretty clear on how to avoid common pitfalls. Reference: When Your Incentive System Backfires by Vijay Govindarajan and Srikanth Srinivas (2013).

Inverted Apprenticeships: Fine, this one’s mine - with ace ethnographer and coauthor Callen Anthony at NYU Stern. In our original research, published just last August, we showed that when new technologies arrive, senior experts can rearrange their working relationships with novices to “mooch” off them as they engage with new technologies. Unfortunately this is rarely a winning equation for the novice in the wild, but we found one clear approach to this that left the novice and the expert better off in their skills with the new technology. Reference: Inverted Apprenticeship, Matthew Beane and Callen Anthony, 2023.

Now, work it all into a plan

So fine, there’s an incomplete menu of scalpel-like tactics to carve out an elegant genAI learning journey for your organization. Given that 2x2 you can go find more that suit your situation.

But that’s not enough. Just having the right instruments on the tray doesn’t mean you can get to work: you need a plan. Each of these tactics needs to be slotted into a coherent process - where each step is more valuable because of the results from the previous one. There, my colleague Paul Leonardi wrote a Harvard Business Review article last year introducing his STEP framework. Here’s a direct rip from his piece:

1. segmenting tasks for either AI automation or AI augmentation;

2. transitioning tasks across work roles;

3. educating workers to take advantage of AI’s evolving capabilities and to acquire new skills that their changing jobs require; and

4. evaluating performance to reflect employees’ learning and the help they give others.

Hard to do any better there - if anyone on the planet can synthesize five decades of research on how technology actually gets implemented in organizations, it’s Paul. And he tuned it to generative AI in 2023! Way ahead of schedule.

A complementary resource here is a solid understanding of how change happens in organizations, and how leaders can foster it. Organizations are built to deliver reliable results on known problems, and inertia (i.e., “best practice”, culture, etc.) will keep them trying to do that even in the face of an existential threat or discontinuous opportunity. They have to get woken up, motivated, see improvement, then settle back down. John Kotter’s 8 step process has been the platinum standard here for decades. I dare you - look at a real change that stuck in an organization - a change that mattered - and you’ll see these steps behind it.

So the next time you see blanket ignorance, bans, or all-hands frenzy in response to genAI - or get pressure to head in these directions, trot out this 2x2, find the right tactics for your situation, string them together in a healthy sequence, and get to work helping your organization adapt. No guarantee of success, of course - these are discontinuous times - but at least you’ll be sitting down to dinner with a full set of utensils.

ps: everything above is false.

Sort of.

Why? genAI itself.

The logic here has a creeping inevitability: we can use genAI in prior tasks, and that changes how we do them and the results we get. So… all those tactics can be performed differently with genAI in the loop. Sometimes to the tune of improved or degraded results, but other times to the tune of results we couldn’t get before - that we didn’t think were possible. How? Nobody knows. Seriously.

Some individuals, groups, and organizations out there are finding genAI-infused ways of doing skunk works, incentive schemes, open innovation, and customer discovery. But those successes are partial, at best, and nobody (as far as I know) is studying them - even inside the organizations in which they’re being discovered!

This is why I waved a big red flag around L&D as a critical function, and why I, Ethan, Paul, and others are saying all bets are off, unless we experiment with new ways of working.

So go ahead - turn to the traditional playbook that can fill out that 2x2. It’s certainly better than the alternative. But be on the lookout for surprises. Deviance. Nonsensical stuff. It may well unlock a new way to help organizations learn in this age of intelligent machines.

Think of your most valuable skill. The thing you can do reliably, under pressure - and that looks like magic to those nearby.

How did you learn it?

No matter who you are, no matter where you live, no matter the skill, a core part of the answer is practice. You stunk at it in the beginning. You tried and tried and tried, made many mistakes as you went, and, slowly, you started to get better. You put in the time, plain and simple. This just kept going, the more you leaned into it - maybe well past what you thought was possible. There’s a mountain of research to back you up here, and that explores the rich tapestry of experience required for more and less productive practice, when it comes to skill. If you wanted to start and stop at one person’s research to understand this, it would be hard to do much better than K Anders Ericsson, a psychologist who studied expert performance in a variety of domains, and literally gave name to “deliberate practice”, a technique that’s now standard operating procedure for elite athletes, musicians and their performance coaches the world around. Anyone who has been taught that visualization and mental rehearsal are a good idea, who’s read that practicing one thing at a time, with intention, will get you farther, faster - or anyone who’s actually read Malcolm Gladwell’s “Outliers” owes their insights to Ericsson.

I reencountered him - and discovered dozens of other researchers, across 13 disciplines - as I pulled together the world’s research on skill development for my upcoming book. They speak with one voice: the “how” matters quite a lot for practice. In the language of my book, we need healthy challenge - the right kinds of difficulty, experienced in the right ways - and healthy complexity - the right kind of engagement with the broader context for our tasks. But to put the details to the side, it’s clear: there’s no way to “git gud” without practice.

If that were the whole story, we could all build the skill we needed to handle the wild world of work we’re building for ourselves. We’d have the internet, YouTube, ChatGPT, and could get to work bootstrapping ourselves to readiness.

Yet we’d fail. In fact, we couldn’t even start.

Let me give you a window into the fuller - more human - picture of skill by honoring Doug.

RIP, Doug Michel - my competitor, mentor, and friend

Rewind time to three years ago, right in the middle of the pandemic. I got an Oculus virtual reality headset for my birthday and my wife happened to pick up an indoor spin bike from her gym that was going out of business (dang Covid). I *hated* cycling - I'm a runner and have never been an athlete - but right away I bought a bluetooth cadence sensor for the bike, one that would pair with the headset. Why? When I was living in Boston, I had tried a virtual reality cycling system from a startup called VirZoom. Turns out, they survived, found a market, and were doing fine. Cycling in a virtual race - or along roads in google maps - was the main draw for most VirZoom users. Not for me.

I played a *lot* of videogames as a kid, and while I toned it down as an adult, I never stopped. So I remembered that VirZoom also had simple, somewhat cartoony games. Those were fun. And my foot was hurt.

So I hopped on, and went straight for the fun - pedal and I got to fly as a pegasus, chase after bandits on horseback, zoom along a riverbed with a helicopter, or get in a tank battle. And boy, tanks were REALLY fun! Especially because you could compete against another human being.

I started to do it a couple times a week, huffing and puffing, getting shot at, and so on. Whenever I got a multiplayer match, I'd get trashed. People were so *accurate* from such a long way, and -wait, what- they'd always have some magical shield on right as I came over a hill and... dang. Game over.

But again, the fun kept me in. I certainly was working hard!

So then one day I realized there were leaderboards for all these games! And I was... in 119th place?! Wait a minute. Wait a MINUTE. Who's in first, and their score is... WHAT?

That's when I thought it was impossible. Literally. I saw a score that was like 27,000 for a 30 minute solo tanks run. I was getting say 11,000 working as hard as I possibly could? They must have changed the game, I thought. Like there was an old sensor that let you go 2x faster, or tanks were worth more, or whatever. So fine, I can't ever get into the top ten.

But let me try for the top 30. That's what I told myself. So I started putting in the time.

And that's when my scores started creeping up. No big revelation, no fancy tricks, just showing up four, then five days a week and working hard, watching and trying to figure out... WHAT. ARE. THEY. DOING.

There were no tutorials, no help. So I was improving pretty slowly. Matches were hard to come by - I didn’t know when anyone would be on!

So finally, in frustration, I reached out to Doug (his username was bakerbiker) on an old, janky user forum. Doug was in the top ten, and climbing. An avid player. He beat me by big margins, every time. So I was a little intimidated, but I had to know: how in the heck do you get that kind of score?

His (friendly) answer: I don't know! Just play. Get the pickups. Don't get hit. Hit stuff. Get good. So… yeah, not too helpful. That’s the way with so much skilled work - much of it just isn’t explainable in language, even if we tried. This is at the heart of Polanyi’s and Moravec’s paradoxes - a basic barrier to development in AI, not just human skill development.

But that was the beginning of him coaching me. We’d play. He’d win. We’d discuss. I’d ask. We’d schedule times to play. We chatted about other players.

And my scores kept creeping up. I got in a LOT better shape - maybe the best of my life, I started to realize - and I started to figure some stuff out. Like how to orient myself so I always had a clear shot, and that there was a pickup that let you fire faster.

And then one day I beat Doug! He warmly congratulated me. Said it was a great workout. Mentioned how much I’d improved. Said he’d become better as I had. Wow. I felt so proud, and he was delighted. It was back and forth from there on in. Then we started to reach out to other players. To assemble a dm group of devotees, eager to battle it out. Soon we had Crystal. Leigh. Superlith. Jazmin. Kathy. BillyDWolf. DrSteve. Paul. Even Eric, the CEO of VirZoom! We all chatted between matches - about the same stuff. Tactics. Goals. And through all that friendly competition I pushed myself to do better and better. The day I hit #1 in all solo-play categories, everyone congratulated me. Same when I topped the head-to-head combat charts. There was no trick. There was no man behind the curtain.

Except for maybe Doug. Beyond the bond that gave me a sparring partner and my motivation to improve, he was the whole group’s cheerleader, our glue. Always up for a match, welcoming to newbies, cheery and grateful for the workout, whether he won or lost.

Then we lost Doug.

His wife dm’d us via his account one Sunday late last year. With a heavy heart, she let us know he’d passed away the day before. I had played him that morning! We were all in shock. Devastated, really. By this point, we all knew each other well. We knew Doug founded and ran Columbia Bread, basically the first successful bakery that catered to people with gluten sensitivities. And he wrote and illustrated a wonderful and strange set of comics. Sue told us she wanted us to know how much the Tank Nuts (the name that Crystal gave us) meant to him. We sent her and her family a custom condolence package. A few of us still fly his flag in the game today (you can select a custom flag for your tank as you win more games). When a match is near over, we often fire a shot over each other’s heads - a salute that Doug invented. We talk about how we miss him.

We all got better because of Doug. As players and as people.

We need healthy connection to learn

We often celebrate leaders who received The Devil Wears Prada–style mentorship - isolation, lack of direction, harsh critique - but these are typically the harbinger of poor skills development. And all too often, we think of skills in individualistic, egocentric terms: all you need is a good head and nimble hands. That masks the fact that we simply can’t get healthy challenge and complexity without the kind of healthy human connection that Doug and I enjoyed. So, if challenge and complexity are more about the “how” of skill, connection is more about the “why” - more to do with a rich, complex, and very human landscape built on warm bonds of trust and respect. So say the fields of psychology, education, and management, but recent developments in neurobiology and artificial intelligence lend even more credence to the idea, too.

This has an externally focused aspect that’s about practical opportunity: Want a chance to work at all, let alone get access to healthy challenge and complexity? Good luck getting it on your own. One or more people have the power to admit you to the party, let you stay, and lend support. Without my connection with Doug, I wouldn’t have even gotten a steady stream of practice opportunities. Zero chance I’d have improved like I did. Beyond simple access, there’s the question of advice and guidance. Remember, the first time I asked Doug how to get a high score, his answer was “I don’t know.” But he knew a lot of things, even if many of them were in-the-bones skill. He just didn’t know, like, or trust me yet - well enough to try to messily explain his experience, give me feedback, and describe a technique or two. I hadn’t earned the privilege.

But there’s also a more internally oriented aspect that’s about personal meaning: we get motivation for our work when it builds respect and trust with those who share our values. These are questions of the heart, like “Do I feel connected?” or “Do I feel motivated to gain the respect and trust from those I aspire to become?” We often dismiss these as unconnected with hard-nosed skill and results, when in fact they’re closer to the main event. I looked up to Doug. In the beginning, this was strictly numbers-based: his scores seemed stratospheric, and I couldn’t quite believe I had the good fortune of finding him on the user forum. At the same time, it was pretty clear we shared some core values: he was kind, a little wacky, inclusive, and of course really cared about the game. I wanted to earn his respect and trust. But he also told me later it was a real shot in the arm that I was so engaged with him - it meant a lot to help someone, to see them improve, and to grow a small community of practitioners.

The human connection was *why* we both engaged in the activity, really. Yes, we wanted to play a game, and get better. But the animating force - and the way it unfolded - was our evolving bond.

No, this doesn’t mean ending wfh and genAI

I tend to be conservative on remote work. I happen to believe that it can’t possibly equal the productivity of in-person work… And then leadership and mentoring: I would not be where I am today had my bosses not been able to see me perform in some physical space. I’m convinced of that.
- Bob Iger, CEO, Disney; a16z podcast, Jan 6, 2023

Now that news of my book is out there, I’m giving more talks about it to groups of executives, professionals, policymakers, and so on. These are a delight, a privilege, and an education, all wrapped into one.

One lesson I’ve learned is that leaders are very worried about the loss of human connection associated with remote work. My findings stoke that fire rather dramatically - understanding the specific relationship between human connection and skill development gives them a new vocabulary to articulate what was only a creeping suspicion before. Then I wake them up to another central finding in my work: that we’re taking advantage of intelligent technologies like robotics and genAI in ways that allow experts to get better results with less help from novices. And that novices are building less skill as a result. That catches most completely off guard: not an issue they’d had on their radar.

This is about when they really start to speak up about fighting for human connection. This isn’t just practical, really. Yes, they see a dramatic, multifaceted threat to informal skill development throughout their organizations. That’s a dramatic issue for performance and adaptability. But it’s also a direct threat to their values. They came up through a system of collaborative struggle with experts. Looking up to them. Learning vicariously as they were included, bit by bit. Earning their trust and respect. And as they became experts themselves, paying it forward to novices. This feels healthy, right, and proper. A beautiful system. They react pretty violently to seeing it under threat. And many of them grab for Bob Iger’s solution pretty quickly.

As I was wrapping up a talk, one CEO summarized this neatly: “you’ve just given me all the ammunition I need to make everyone come back in the office.”

We all had a *very* interesting and valuable conversation after that.

It settled on the same reality we all have, more or less: remote work is here to stay. So are intelligent technologies. A wholesale return to the way we used to do things would be destructive, but it’s also not an option.

Finding positive needles in the negative haystack

So the more useful question is how do we get the benefits of remote work and intelligent technologies while improving on the past? Not just get back to where we were. That would be great, in many circumstances, but let’s face it, pre-pandemic work reality wasn’t ideal. So all this disruption provides unique opportunity for a species that - collectively - prefers predictability and stasis. In the language of this post, how could we enhance bonds of trust and respect between experts and novices because of remote work and intelligent technologies like generative AI?

Answering that question in detail is the job of the back third of my book, but a simple answer is literally sitting right in front of you.

I never met Doug in person. Didn’t even know what he looked like for a year and a half. And that was the most important year and a half, when it came to our bond and my skill. In fact, I’ve never met *any* of the Tank Nuts in person. We live all around the world! Those bonds mean a lot to us, and have allowed us all to become far better players than we would otherwise. And none of them would be possible without the technologies behind remote work.

We are mostly failing at improving healthy connection because of technological disruption. Many of us have firsthand, painful knowledge of junior employees languishing for lack of mentorship in a work from home environment. But another core finding in my research tells you one thing for sure: somewhere, somehow, a small subset of us are finding systematic ways to enhance human connection in spite of these barriers. And in a few precious cases, technology is how they’re doing it. Often it’s the very technology that’s at the heart of the trouble for the rest of us!

Our urgent task is to find these systematic successes and learn from them. This is what I did in robotic surgery, and have done in every study since. Most of the time they’ll be partial. Other times they’ll involve unacceptable deviance. Nobody figures out the new best way in one go. The key is they have to be working in very different settings, with different specific technologies, different occupations, kinds of work, industries, geographies, and so on. The more broadly they appear and help people cultivate healthy connection, the more we can trust them.

In my strong opinion - built over more than a decade of research on skill and work involving intelligent machines - this will work if we anchor our search and redesign on what we know about how we build skill and get results. From the point of view of my book, we know we need healthy connection. The book provides a fine-grained checklist on what that looks like in action. We need to take a hard look at our situations and consider the facts: where and how are we handling technology in a way that gets results, but hurts connection? Where are we handling it in ways that helps connection, but compromises results? Where are we handling it in ways that improve both? And, if these are systematic, how can we share these unintentional gains productively with the organizations and billions of working adults who need healthy connection now, more than ever?

This is our urgent quest. It’s going to require hard work. Sacrifice. Investment. Inertia is in favor of productivity over connection, and that’s a death knell for skill, right at a time when we need it most. Let’s get to work.

ps: if now you’re curious about getting a great workout in VR tank battles, here’s a narrated YouTube introduction I recorded about two years ago. Let me know if you want to give it a try - the Tank Nuts will show you a good time!

Should my kids learn to code? Can I keep up with my coworkers? Can this person do the job I’m hiring them for? Should I trust this company with my problem? Do we have the workforce we need for our industry’s problems?

The very public advent of generative AI - and the multifaceted assistance it can provide - means questions like these are being asked and answered anew, around the globe. In case you need to get up to speed on the apparent disruption, I recommend this short video of a naive user - someone with no training in software development - using conversational English prompts to create a functional game that will run via the internet on their mobile phone. I watched many of my Master’s students do something similar as they analyzed and plotted a large dataset via python - most with no prior coding experience. We’ve seen comparable examples with reading, writing, and basic math, but also with web design, graphic design, scientific research, customer service, statistics, project planning, and medicine, just to name a few.

New results abound, and we want them. In fact they’re probably more important than ever, given the complex problems we’re facing. But the way to do our old tasks with these new tools is now a bit murky, and we have a thicket of unanswered, critical questions:

• Does genAI mean we’ve arrived in an era where knowing exactly how to do the work is the equivalent of knowing arithmetic by hand when a calculator is available? When does it not mean that?

• Does getting better results with genAI mean we’re more skilled? Less skilled?

• What skills do we really need to get results, and how can we trust if someone has them?

• How do you build and maintain skills, if you’re relying on genAI?

Absent clear answers, we’re all making bets.

Parents, deciding on school options. Recruiters, deciding on candidates. Educational administrators and EdTech providers, deciding on curricula. Clients, deciding on law firms. Patients, deciding on doctors. Learning & Development professionals, deciding on corporate training. Teachers, deciding on class design. Administrators and executives, deciding on strategic investments. Politicians, deciding on policy. Even kids, looking at code, writing, and art they get just by prompting for it in English - some are asking and answering for themselves, too.

Long before we had the science to explain it, we’ve known that the right skills beget results, power, and opportunity. So we’ve worked hard to provide up-to-date paths for skill development, because they would help learners contribute in ways that meet society’s needs. This also often gives them a personal leg up - in pay, dignity, and job quality. While it’s as old as the human species, this kind of adaptation is clearly the story of all classic education, vocational training, higher education, corporate L&D and formal apprenticeship programs: the world identifies outcomes it’s after, the skills required to achieve them, and sets about helping humans build those skills and put them to work.

None of that can keep up with what’s happening right now.

I’ve been studying the implications of intelligent technology for skill development for over ten years now. Before that, my work was likewise focused on skill development and learning for another decade. And to write my upcoming book I familiarized myself with decades of research on skill development spanning over a dozen disciplines and a hundred and fifty years. Based on all that, I can confidently say the skills target is moving and changing at unprecedented speed - and we’re not ready for it. We’ve got to rethink skill right away, or many of our bets are going to turn up losers.

Skill at the Jagged Frontier

Let me ground this all out by referring to the now deservedly-well-known “Jagged Frontier” paper. The findings here are, objectively, bananas: consultants were more productive and did better quality work by wide margins when using genAI, and those below the average skills distribution improved their performance 2.5 times as much as those with above average skill. If measured by results, all of these consultants got better, but the worst ones saw the biggest boost (important to note that a *wonderful* study of Kenyan entrepreneurs showed the opposite effect).

Some have claimed this means those consultants are, by definition, more skilled. Skill is that which allows you to get results, and users’ results are better, therefore they have more skill, Q.E.D.. Ethan Mollick and I had an interesting and informative conversation about this issue and the Jagged Frontier paper on Twitter, where he started out with this view. And it definitely has some truth to it. But not enough for games with consequences. Yet have no doubt: people are using this logic to apply and select for jobs or work gigs, make decisions about reskilling their workforce and training budgets, and predict the socioeconomic impact of genAI.

There are at least four serious problems with this line of thinking:

• It conflates individual skills with bundles of skill (aka jobs, activities). Being skilled at writing a research brief is *very* different than being a skilled consultant. In my fall experiment, my Master’s students hit their stride producing code, but really struggled with completing their first individual assignment. Why? They had to get their code into GitHub codespaces - a cloud-based development environment that lets users write and run code from their web browser. Many folks would call all of this “coding in python” and miss the stark difference between producing code on your local machine and working through platforms. The professionals who curate oNet - the U.S. government’s occupational dictionary of tasks for all known occupations - work hard to avoid this trap, but most of us gloss right over it. Most work takes a bundle of skills.

• It presumes away surprise, change, and complexity. To riff off Mike Tyson, everyone’s got a skill until they get punched in the mouth. Sometimes writing a memo is just writing a memo, but other times we need to fly to Bangalore to interview an expert because their internet and phone are down, and the client bought the memo for their expertise. Sometimes a crossing guard has to deal with a 500-person Bike Life group, right when school lets out and the kids need to get home. Sometimes reality intrudes - it’s complex and dynamic. And we expect people to handle quite a bit of it. Until we have practiced using genAI to handle these predictable surprises, our apparent new “skill” is shiny but brittle.

• It ignores task characteristics. We can trust genAI too much. And it hallucinates. None of that is okay on a task where failure is not an option, or when technology access is problematic, for example. In a robotic partial nephrectomy (removing a cancerous tumor from a kidney), if you knick the iliac artery, your patient could be dead on the table in three to five minutes. What do you have to do to solve this problem? Undock the robot from the patient - which involves removing four laparoscopic instruments and backing the thousand-pound apparatus six feet away - page a vascular surgeon, gown and glove up, and convert to open surgery via a large, manual incision. The robot is no longer useful - it’s a hindrance. Many other task characteristics matter here - time pressure, irreversibility of decisions, task feedback… we have to take many of them into account to assess skill.

• It ignores learning. Hard to say genAI boosts our skill if our results get better but our broader abilities stagnate or decay through use. I wrote about this last year: AI can dumb you down. Not because you get worse at the thing you’re doing. Because you get better at it. If you get a response you’re satisfied with, much faster, you have much less opportunity to struggle, and to encounter the broader context for your work task through “inefficient” exploration. You likewise lose an opportunity to ask for help or offer it in return. Challenge, complexity, and connection, in the language of my upcoming book. It’s this robust, semi-focused process that allows us to build and maintain valuable skills.

If we take just those four critiques seriously, a lot of the genAI-enabled “skill” out there is spindle-thin, fragile, hyper-specialized, and static. It’s not deeply trustworthy, at least in the way we’re used to trusting skill. It would need to be measured differently. It’s social and economic impacts would be different. We have not studied this - or even modeled it - and we need to, because it really is an empirical fact that hundreds of millions of us are applying this kind of skill in consequential work.

The new wave of skill: Interactional expertise

To put this to the side, however, I think this flood of new “talent” into the marketplace highlights a kind of skill that has always been with us, but will matter much more in the days, months, and years ahead.

A Master’s student of mine unwittingly (and cynically) highlighted it when she said:

“I don’t think this qualifies me to work at a software company, but I guess I could gaslight them into employing me.”

She was referring to the software development skill she’d acquired through the quarter-long, intensive experience her class had just been through. Together they’d gone from having no coding expertise to doing data analytics on their own via python, then working in teams to build a functional, web-based, software tool for technical project managers. This required many hours of staring at and tweaking code, figuring out how to run it, seeing it fail, debugging it. Then stitching it all together and running it live in front of the class. They thought it was impossible, but it worked.

But did they have coding skill? Clearly, in key senses, they did not. My amazing TA Brandon Lepine and I interviewed a representative sample of them after the fact, and as part of that interview we showed them a set of very simple code snippets. A “for loop”. A “dictionary”. Some recursion. Many had no idea what they were looking at, other than “code”. Some saw a few key words they understood, though. “I know that’s a print statement”, some said. Or “that’s a list of data.” If you took ChatGPT away from them, or turned off the code interpreter feature, or asked them to think through a problem computationally, they’d be dead in the water. So there’s your thin, fragile, hyper-specialized, static “skill.”

They did, however, build a lot of a very different kind of skill. Skill that’s about a reasonable understanding of the system of work they were involved in, its vocabulary, its dynamics, its processes and tasks, its norms, roles and responsibilities, and its output. They roughly knew what was going on, and could go through a lot of the motions with some fluency. That's a special and powerful kind of skill - useful for a lot more than gaslighting an employer into hiring you.

Harry Collins called this “interactional expertise.” It’s knowing enough about a context to be “conversant” with local experts. To understand the basics of their work to their satisfaction. This allows outsiders to plausibly interact with insiders. Interactional expertise is what allows managers, journalists, and organizational ethnographers like me to do our jobs. It enables interdisciplinary collaboration and peer review. It lets the public engage with complex issues. It is literally the grease for the cogs of society, science and commerce.

If you use generative AI to do something you couldn’t before - even a little bit - you have an opportunity to build this kind of expertise. Especially if you configure it to push you in that direction (here’s my updated, first-pass how-to on that). You’ll be in a position to build better, more productive relationships with experts outside your areas of solid skill, and to identify places you’d like to go in your skills journey.

But the benefits of interactional expertise are about to mushroom, Oppenheimer-style.

Skill is dead, long live skill

David Autor - one of my intellectual heroes, an economist at MIT - recently wrote a marvelous piece for Noemi (and NBER), tantalizingly titled “How AI could actually help rebuild the middle class.” It’s a *must* read, full stop. Tl;dr, however? Our society has concentrated wealth and power in the hands of those who can make expert decisions, and generative AI could break all that in fruitful ways for the middle class. In David’s words, the genAI we’ll soon have will likely:

…support and supplement judgment, thus enabling a larger set of non-elite workers to engage in high-stakes decision-making. It would simultaneously temper the monopoly power that doctors hold over medical care, lawyers over document production, software engineers over computer code, professors over undergraduate education, etc.

Sounds incredible, but it’s quite plausible. If hard sci-fi had an equivalent in labor economics, this would be it. Lots of social science predicts elite experts won’t go quietly into the good night, but in certain senses if these systems get really good, we should expect them to have no choice. Urgent medical care in rural areas. Legal help in impoverished ones. Technical tutoring and design help for kids with no nearby instructors. Against all the research on sociology of work and organizing, occupations, occupational jurisdiction, and so on: it’s going to happen, at least in pockets.

But to take advantage of this newly-available, nearly-free elite judgement, users will need… you guessed it: interactional expertise!

You have to know roughly the game you’re playing - at least a few moves deep - to put forward a cogent request to your elite-expert-in-a-box. You have to express your problem in terms that would allow it to make a precise judgment. Sure, it will ask clarifying questions to try to help you, but if you have interactional expertise in a given domain, you’ll be able to get great results much, much faster. That will in part be because you’ll be able to convey the output to real people in the real world as you try to get value from them. You’ll have to explain. Convince. Enroll. Rebut. Modify. Learn. If you’re only thin, fragile, hyper-specialized, and static, you’re toast.

Interactional expertise is the new metaskill that you’ll need to program GPT-4+ level systems in ways that loosen the chokehold of elite experts on the economy and your life. Not that many of those experts want to have their hands around your metaphorical neck, mind you - doctors, lawyers, plumbers, teachers - they’re all super stressed, and are well aware they can’t help enough people fast enough. I think, in part, that’s why we’ll see some surprising movement on this front - the benefit to otherwise underserved people will be too significant, too quickly.

Even if our prior social structures and ways of working remain intact - even if elite expert judgment still wins the day - interactional expertise will afford the rest of us a great deal more agency in our work and lives. Being a conversant generalist is the new superskill, and if you use it right, genAI can help you develop it faster than ever before.

If you’re reading this, you’re probably fine.

You’ve been up on genAI for a while now. You followed ChatGPT back before it was cool - you probably remember when the “code interpreter” feature came out, and you might have even used ChatGPT 3. You quickly signed up for Google’s new Gemini Advanced - maybe even on the day it was announced in early February. And you likely know that just this Monday, Anthropic released Claude 3 Opus, a model that “outperforms” ChatGPT 4. You *might* even know what those airquotes were about: Anthropic benchmarked Claude 3 against ChatGPT 4 at release, meaning they didn’t compare against OpenAI’s latest, most improved model. In a highly competitive market, the sizzle sometimes overpowers the steak.

And if you know all this, you’ve also tried these systems out. You’ve gotten first drafts. Edits. You’ve written a funny, likely terrible poem or story (or twelve). You’ve created songs with Suno. You’ve generated code. You’ve asked for online research, and for synthesis of the results. You might even be that person who helps friends and colleagues get up to speed. You know enough to be helpful to a novice - getting them oriented, addressing questions and fears, teaching them the basics, and getting them started. And if you’re a loyal reader, you might have even taken the “try the impossible” challenge I issued a couple of months ago: from a cold start, use genAI to attempt a task you literally thought you couldn’t perform, no matter how hard you tried. One way or another, you’ve learned a lot - about what this technology can do, what it can’t, how it can mislead or confabulate, and how to handle that to get the results you need.

If you did all this and you swap your labor for money, you likely do work that is significantly “exposed” to generative AI - in other words you can use genAI to get better, faster results on a healthy chunk of your work tasks. Given that genAI can’t interact with the physical world, these tasks are overwhelmingly cognitive - produce writing, analyze data, write code, create or analyze images, reconsider or craft concepts. Imagine anything you could do remotely and you’ve got your arms around this kind of task.

But I can also predict a few other things about you. The first follows pretty directly from the prior paragraph: you probably earn(ed) a pretty good living. The kinds of tasks above - the kinds that genAI can help with - are on the cognitive/social side of the range of human skills. Those are the skills that our society pays for, and it pays a lot more for them in many cases. Most sensorimotor skills just don’t get the same pay. Yes, the middle class that handled more routine cognitive work has been hollowed out over the last forty years or so, but those who are left are dealing with real, complex problems. If you’re in finance, HR, marketing, or some other internal function, your job might be a bit too routine for your taste, but it requires judgement, emotional intelligence, and more. So, fine, you’re doing okay financially, and you’re in what many labor economists would call a “high skill” job (I strongly object, but that’s another story).

Next: you’ve got discretionary time. Nobody’s studied this yet, but “getting” the relatively unrefined genAI we’ve been given requires a lot of experimentation, and I think a lot of that is lonely failure. Trying things out, getting confused and frustrated, ending up without an answer and giving up, finding out a long chain of prompting isn’t really worth it for the results you got, searching online for tips and finding underinformed hype, keeping up on the news of it all, barely pausing to wonder who else is doing this, let alone connect with them… all in exchange for a few eurekas and a small pile of insights that allow you to be reasonably efficient with it today. It’s not obvious you’ve come out ahead on pure productivity terms. Most haven’t. In fact late last year I used this logic to argue that most of us should probably cool our jets on genAI. It’s a time sink, but let’s face it, you’ve got the time to burn.

Relatedly: you’re a bit of a deviant. Even now, genAI is like electricity and a large electric motor in a previously nonelectrified home: you can get a lot of value out of it, but you’ve got to be willing to do something that you’ve never seen before, and you can’t guarantee it’ll be effective, appropriate, or safe. In the midst of your efforts to “get” your new electric motor there would probably be a moment (or twelve) where if someone walked in on you, they’d see a Rube-Goldberg style contraption made of belts, pulleys, boards, braces, and maybe a feather duster. You would likely end up with a few literal scars or burns to show for your efforts, and your family would at best tolerate the disruptive panjandrum in your living room. All en-route to an automated dusting device. Obviously using genAI is a lot more straightforward than all this, but it’s metaphorically the same: you have to be willing to bend the rules and strain propriety to learn your way to productivity. This mixture of deviance and progress goes hand in glove with “shadow learning”: the kind of skill development I uncovered in my research on robotic surgery.

To knock off a few more: you’ve probably got solid internet access, are based in North America or Europe, and can think and work well in English. The internet access bit might seem obvious, but that’s because you’ve got it. Less than 1/3 of the world’s population enjoys the same privilege. Then there’s the geography bit: OpenAI, Google, and Anthropic have made their systems available first in the United States and second in Canada and Europe. Are you a world-renowned economist who wants to use Claude 3 Opus for research on release day, but live in Canada? Sorry. Some of the reason is doubtless geopolitical. But some of it is linguistic. These foundation models are trained on the internet, but really if you look under the hood they’re trained on masses of text in English. All serious firms are working their way out from there, but their internal cultures, training data, marketing, demonstrations, support capabilities and so on are all rooted in English.

So, to sum up: you’re probably do work that’s significantly “exposed” to genAI, earn(ed) a good living, have discretionary time, bend the rules and defy norms more than most, have solid internet access, are based in North America, and speak English.

Most humans don’t.

And that means - when it comes to the skills needed for the wild new world of work we’re inhabiting - you’re probably going to win, and they’re probably going to lose.

This is the specter of skills inequality.

The available science suggests this is fractal - occurring on the individual, group, organization, industry, culture, and national levels. As an individual you have the resources needed to engage with genAI, you have a much better shot at building skill with it than someone who doesn’t. And if it enhances your productivity, you’re going to get access to that next valuable opportunity more, better, faster than they can. Sometimes, of course, you’ll lose. Fall on your face. Waste your time. Hurt someone. So it’s a bit tricky, but—on average—you’ll seize opportunity.

But things get trickier from there. What makes for skills inequality at the individual level might have different effects at other levels. I found this in my study of robotic surgical training, and called it a “Matthew effect for skill”: the better you did with the technology, the more practice you got, and that came at your peers’ expense. They build less skill. So that group - and, by proxy, that occupation - saw severe capability limitations over time. You might see something comparable if everyone (or even most everyone) tried to use genAI all at once. Semi-focused, variable and spiky productivity, and the performance at the group level could seem simply meh.

Skills inequality doesn’t necessarily mean trouble for groups and organizations. If a few people or groups race ahead by design and people can plan around that, the collective could learn from their experiments, optimize them, and then race ahead in unison. In a sense this is what formal R&D departments do in large organizations. Learning and development professionals and functions can play a critical role in enabling this kind of thing. Social systems are fiendishly complex, so there will be other, surprising ways that skills inequality is a tide that raises all boats.

Sociology trumps agency

But the sociologist in me doesn’t operate on hope, positive exceptions, or possibilities. It sees social forces at work. Situations, practices, tools, institutions, and cultures that make certain outcomes more likely for anyone who gets involved. Deprive someone of resources like money and, on average, they start to behave and think like a poor person. Put someone in a social network with a certain set of political beliefs and, on average, they’ll adopt them. Give us technology that allows us to get rapid productivity gains if we reduce novice involvement in the work? On average, we’ll do it. From this point of view, the “safe” prediction from all the research I’m aware of—mine and others’—is that we will continue to seek immediate productivity from genAI and other intelligent technologies. As individuals succeed with less help, the collaborative bonds between those who can and can’t will fray. We will degrade healthy challenge, complexity, and connection - the necessary components of skill development that comprise the skill code at the core of my upcoming book. The rare few who find shadow learning practices or who are lucky enough to avoid these traps will race ahead with far more skill than the rest of us.

We worry about income and wealth inequality these days. Serious problems. But right now, we are also racing down a road toward skill inequality, and the sociologist in me doesn’t see any forces arising to change that on a mass scale.

We do not want to live in that future. First off, individuals who hit a wall on skill development will hit a wall on their careers, their income, and their quality of life. For most surgeons, this might mean getting blocked from top-tier hospitals, limits to income, and the stress of being behind the times—while a few race ahead to far more prestige, more miraculous patient “saves,” and dramatically higher income. But skill inequality goes way deeper and darker than it will in high-status, highly paid professions. After spending three years interviewing hundreds of warehouse workers with my team, I can tell you firsthand that the aphorism attributed to William Gibson (“the future is already here—it’s just not evenly distributed”) is just as applicable here as in robotic surgery: this darker future is already here, and it’s heavily concentrated in low-wage, entry-level, repetitive work. These folks are often temporary workers. No benefits, no job security. Barely any training, practically zero mentorship. And warehousing corporations invest massive resources to aggressively deskill the jobs these folks occupy. Industrial engineers walk the floors, doing detailed time-motion studies to figure out how they could improve productivity. For a warehouse? That’s how many items the building can process in a day, times a “defect” rate—or the percentage of units out of a hundred that don’t get damaged, badly labeled, stolen, and so on. And what’s the best way to increase throughput times quality? Reduce “skilled touches.” Those industrial engineers know that every time a human has to deal with an item, there’s a chance for an error. And that the more skill it takes to handle that item, the greater the chance of the error is. So they are continually redesigning the work to extract skill requirements from the job and quantify the output—not just reduce the number of times a human has to interact with the product. It’s only one more step to the stark conclusion: the longer that a worker stays in jobs like this, the less skill they can expect to have over time. Their skill degrades.

That’s degrading, too. A lot of folks, according to one worker, get to a place where they

“are content with just doing the grunt work. I mean because a lot of these pick and pack places—it’s like high school, do you know what I mean? You’d rather just stay in your lane in the palletizing and don’t even get involved with the people up there [in other areas].”

So long, healthy challenge and complexity. So long to healthy connection, too. Said another one:

“Honestly, I can’t [tell you about what my coworkers are like] because I don’t talk to anybody there. We’re trying to go so fast to make our numbers.”

And this last worker tells us the nasty conclusion—the longer you stay in a job like that, the less likely you are to look for something better:

“there’s a lot of warehouse jobs that you see people after being there for a while, and I just can tell they’re drained, tired all the time, and just fed up of it but [they] don’t really know anything else.”

Here and in many comparable places throughout the economy, we really are destroying human potential on a mass scale in the name of techno-productivity. The rare few shadow learners that my team and I found in warehouses all around the US prove that we can build rich, valuable skill just about anywhere. Deskilling, degrading work is always optional. On top of that, these workers’ innovations show us that favoring skills over pure productivity isn’t an act of charity or empathy alone. Insisting on skill development and productivity offers huge competitive business advantages, too—and opportunity costs if we stick with the skill-sacrificing status quo.

A world where the skill code is healthy and vibrant will not write itself. Without a focused, massive investment, we’re going to get more of the skills future I saw unfolding for most workers in warehouses. To pull this off—to stop the dark sociological skills prediction from coming true—we must build a new, global, AI-enabled infrastructure to strengthen these foundations of skill instead of undermining them. On that score, we need to take the very technologies we’re concerned about—robotics, AI, cameras, the internet, mobile devices, and now large language models like ChatGPT—and use them to build the infrastructure for skill we need in the twenty-first century. And those with power in organizations need to weave this new infrastructure into everyday operations so they’re more powerful than they would be otherwise. This is a future that brings humanity and technology into an even tighter dance where challenge, complexity, and connection thrive.

The (Healthy) Future of Skill is Chimeric

In this new world, we would have a network of human experts, novices, and AI, focused on building human and AI capability right in the middle of work. This could begin with AI-assisted matches between experts and novices who happen not to work in the same physical space or organization. Firms are already doing this in coarse ways, and they’re starting to blend genAI into their solutions. But a healthy future for skill will mean a comprehensive new fabric for the expert-novice connection—where simply by engaging with it, both humans and AI learn faster than they could on their own, enhancing human relationships and our sense of fulfillment along the way. A lot of the tools to enable this are already on the table: human-computer interaction researchers like Kurt Vanlehn and Michelene Chi at the University of Arizona have shown us interactive, automated systems that provide powerful assistance and even collaborative glue as learners seek skill. Sometimes these help us build skill more than a human tutor would—a bounded but real solution to Bloom’s “two sigma” problem (basically that 1:1 tutoring is far better for student learning than 1:many). Generative AI offers significant opportunities to enrich and deepen this human-technology partnership, and our collective skill along with it.

We’ve started calling systems like these chimera. This was originally the name for an ancient Greek creature with a lion’s head, a goat’s body, and a serpent for a tail. A blend of different entities. A system is chimeric when it is neither human nor technological—one that allows us to take full advantage of both in ways that do better than either humans or AI could do alone. An early, clean example of this is in the world of chess. For centuries, humans were best at the game, competing and examining the game with fearsome intensity to advance the state of play. Then, in 1985, computer scientists at Carnegie Mellon started to develop a chess program they called “Chip Test.” But you probably know it by the name that IBM gave it after they took on the project in 1989: Deep Blue. In 1997, it won an exhibition match against Garry Kasparov, the great chess grandmaster. That was it, apparently: computers were best at chess. But it didn’t last. Soon the best chess players were chimera—a human partnering with an AI won against a human alone or an AI alone. The AI could propose a complex tree of moves that had a high probability of success, given the lay of the board. But the human was best at intuiting play styles and taking considered risks. Ultimately AI rose to the top again, but chess isn’t a complex problem compared to real life. When the board, the rules, and the number of players shift in unpredictable ways, chimera will have real staying power.

Some companies are using chimeric systems to help people become more effective in their jobs—like maintaining gas turbines, laying out computer chip designs, operating on cancerous tumors, and even harvesting crops. When designed carefully and deployed well, chimeric solutions scaffold the human to more productivity than either human or technology could achieve alone: subtle, early preventative maintenance that keeps a turbine spinning for less cost, ingenious chip layouts that save power, being sure you’ve gotten all the cancer in an operation, and doing far less damage to a crop as you navigate a combine with hyperhuman precision. But as my research shows, today’s chimera almost never develop that human’s skill in the process. They don’t promote healthy challenge, foster healthy complexity, or facilitate healthy connections between humans. In the short run, using intelligent technologies gives us great productivity; chimeras can sort tomatoes or trade securities like a fiend. But then the humans in that loop forget how to do these things and there is no chimeric skill-building system in place to redirect, utilize, and enhance that person’s intelligence and capability.

We don’t have a digital apprenticeship to replace the analog one we’re losing. To save human ability in an age of intelligent machines, we’ve got to build one. Starting right now.

I wrote the book to help a broad audience save human ability in this age of intelligent machines. Early feedback from people I trust and respect tells me I did my job right: it offers insight and practical help to anyone who picks it up - regardless of skill level, career stage, occupation, or role.

But for all our benefit, today I’m going to focus on one group close to my heart, part of organizations around the world: learning & development.

The professionals in this function curate the human ability that backs up their organizations’ promises to the world. Each of us owns our skills journey, of course, but L&D professionals are experts at what it takes to build relevant, valuable expertise in their context. Accounting, Marketing, Finance, Maintenance, Facilities, Sales, Recruiting, R&D - all these functions do work that keeps an organization running. L&D is special in that it does the work that helps all the other functions stay relevant in changing times. A poor L&D function means all other functions get dull through use. A good one helps all those saws keep the edge they need to handle the next challenge.

If you’ve been through a training, read an FAQ or a job aid, been through a job rotation, been a formal mentor or mentee, or had a performance review, you’ve relied on the systems and processes these experts put in place. When they do their work well, your skills journey is clearer, more motivating, more valuable to those you serve, and helps your organization adapt more effectively. I should know - I spent over a decade in this industry before I went back for my PhD, working at this problem from the outside as a consultant and from the inside as a manager. Professionals in this function are fascinated with how people learn best, and have the skill to make learning mean business.

So when I started to write The Skill Code about three years ago, I believed L&D professionals were key to their organizations’ adaptation to intelligent machines.

I just didn’t realize they would become mission critical.

Like most of us, three years ago I didn’t anticipate the generativeAI tsunami. Yes, I believed intelligent technologies would play a rapidly expanding role in the world of work. My wife and I bet my career on that belief in 2009.

But very few of us had the luck or the foresight to anticipate that we’d see the sudden arrival of a general-purpose technology that was available to billions of us for free - one that could perform a wide array of cognitive tasks very quickly and that interacted via an instantly familiar chat-based interface. Let alone that it would soon handle images, video, and audio, too. Let’s face it: though it’s weird, makes stuff up, and responds to social cues, to a naive user this technology is a shock. It seems intelligent.

The revelation - which I stand by - is now pretty clear: many of us will change the way we do our jobs to accommodate this new cognitive prosthesis. The now well-known “GPTs are GPTs” paper suggests that 80% of us could use generative AI to improve our productivity on 10% of our tasks. Many of us will go for it - staying put will often be like choosing to send faxes after email hit the scene. That’s an appreciable chunk of reskilling for 2.6 billion people. And if we take the combination of genAI and robotics into account, all of those figures are going to expand.

Without bold, immediate help from L&D this will hobble organizations.

Why? Without guidance, some employees will race *far* ahead with generative AI, but their collective sprint will be incoherent and uneven. Some will run in circles. Some will cause harm, or slow others down. Some will make stupendously valuable discoveries. And it will all be obscured in the welter of everyday operations: sales, customer service, perhaps even R&D might improve… a bit. Or key outcomes might just say the same. This is exactly what happened in my field study of robotic surgical training across 18 of the top teaching hospitals in the U.S.. Prior training methods remained in place, and got in the way. If you followed that playbook, you struggled to learn. A few residents found alternative ways to build skill anyway, and they raced ahead. But they hyperspecialized in robotic surgery, and their expanded learning opportunity came at the expense of the others in their learning cohort. Yet all graduates were legally empowered to use the tool when they left, and many did. It’s just that - in the words of one Chair of Urology - most of them “sucked”. So it’s no surprise that from a statistical point of view, robotic surgery was just as effective as prior methods. Analysts couldn’t take skill difference into account.

Things will be dramatically intensified here with generative AI, because unlike a thousand-pound, four-armed surgical robot, you can quickly use genAI for a wide variety of tasks, in a wide variety of ways, and it’s very easy to conceal the fact that you’re using it. People are going to scatter to the four winds with this stuff, and it will be very hard to learn from their successes and failures, let alone focus them.

Of course many, many folks will simply not do much at all, because they don’t know much about the new technology and how they might apply it in their day to day. And Microsoft’s bold play aside, it really hasn’t been integrated into familiar workflows in ways that take advantage of its rapidly expanding capabilities. At best maybe they’ll try something trivial with it, get a moderately interesting result, and settle for that. This variability includes managers, too: even if they got perfect, shared information on the potential upsides and downsides of implementing intelligent technologies (fat chance), they’d take different approaches to redesigning jobs and reskilling workers. The research on disruptive innovation and resource allocation makes it clear: some will implement bold, focused skill development experiments. Most will do a little something. Others will ignore the reskilling and work redesign challenge entirely.

Without top-notch L&D, introducing genAI into work is a bit like yelling fire in a crowded theater: it will tear organizations, teams, and careers apart from the inside. And I shouldn’t say will, actually. This isn’t future tense. It’s happening.

Leaders need to get wise to this, immediately. It doesn’t matter what industry they’re in. Their firm’s strategy is irrelevant. Whether they prioritize their sales teams, R&D function, manufacturing, supply chain team, service delivery, or any other group. They need to make bold, targeted investments in L&D to help their mission-critical functions adapt fast enough to capitalize on new opportunities and cope with new threats.

There’s a dangerous twist, though: the L&D playbook is out of date - follow it and you may well destroy more skill than you build.

Over the last 13 years I’ve found that the skill development game itself has been subtly disrupted from within - via the very intelligent technologies that are at the heart of the opportunity. The short story? The unifying feature of these technologies is that they allow us to self-serve far more than before. So we get productivity from them at the expense of collaborative interaction - most notably between experts and novices. The vast bulk of skill development occurs informally, right in the flow of work, so this is a profound threat, right when we need new skill the most. To add insult to injury, the traditional L&D investment is overwhelmingly weighted on formal solutions (e.g., training, job aids, mentorship) set apart from actual work performance. These need revision and professional attention, of course, but especially given the disruption to natural collaborative patterns, this emphasis is now officially backwards. My 2019 Harvard Business Review article sums up the argument, and I believe has stood the test of time.

All of this adds up to one, clear imperative: CEOs need to quadruple down on L&D as their new, #1 most critical business partner, and hold them to a higher standard than ever.

For the rest of this piece I’m going to get practical, and speak directly to the L&D professionals out there. If you want to show some leadership and renewed relevance in this crucible moment, I recommend the following:

Step 0: Put on your own genAI mask before helping the business

As a matter of professional curiosity, you’ve probably at least tried out ChatGPT. And if you’re browsing around to stay current on AI, you may well have landed on material like Sal Khan’s impressive and inspirational Ted talk which included a demonstration of Khanmigo - an AI-based tutor for kids that has learning theory baked into it. It won’t give you the answer. It’s designed to teach you to fish. A wondrous vision for a complementary human + AI skills future, with proof of the first few steps.

Good for you. Staying one step ahead of those you serve is absolutely necessary to serve as a confident specialist partner in key organization design and management decisions. If you haven’t done this yet, you must do so, stat.

But you have to go deeper. Significantly so. To “get” AI and become a trusted, mission-critical partner for the business, you have to try the impossible: do some real technical work in one of the areas you serve, alone, with only generativeAI at your side.

Seriously. I wrote a post about this recently, and I believe you should treat it as a mandatory. The power, peril, and opportunity with generative AI is partly in how we can use it to do certain tasks incrementally better, but it is also in how we can use it to do things we never could before. Something we wouldn’t have even considered trying. In my post I show - in detail - how I challenged a class of Masters students to build a professional-grade web app for project management. The trick was they didn’t know how to code when we got started. That stopped them from pursuing lots of opportunities, and kept them out of the consideration set for technical project management in particular. Can’t build software? You’re not getting the job.

Let’s face it. You have long had a comparable problem: internal customers often see L&D (and HR) as a “soft” function with limited technical and quant fluency. You don’t “get” them, so from their point of view you can’t really serve them.

You have a golden opportunity to kill both birds with one stone here, just as my Master’s students did: get your own visceral sense of genAI while building credibility with those you seek to help. Only then will you be in a position to help your internal clients make wise decisions about how to redesign work in their areas, who needs what skill to support that change, and how to get it to them. You need to understand how genAI can be an ally - not a hindrance - in all this, too.

Here’s how you can adapt the full assignment I gave my Master’s students to achieve these goals. The first part is a solo act - a mirror to Phase 0 in my assignment. Don’t ask for any help, don’t google anything. Just try - and judge yourself by the criteria I list in that assignment: candor, grit, and creativity:

1. Choose a business unit focus, and design a synthetic dataset to analyze. Decide on a few of your clients’ key metrics (e.g., Sales: leads, conversion rates, revenue per customer, product performance. Software engineering: bugs, deployment frequency, velocity, backlog, dev satisfaction). Then decide on data types (numerical, categorical, text-based). Your choices should outline some part of the data that their function treats as mission-critical. Key to note: only one person in your function needs to create and validate the dataset and analysis tasks. If you’re first, congratulations! Share it - and your process learnings - with others taking this challenge. The point is to get to analyze (starting at step 5).

2. Create data. Since you shouldn’t share your company’s real data, program genAI (Gemini Advanced or ChatGPT 4.0 or Bing in creative mode) to create synthetic data for you to analyze. Yes, programming proceeds in natural language now:

   “Generate a synthetic dataset with 500 rows for a mid-sized software company's [sales] department. Include fields for [list specific fields you decided on], and ensure the data reflects [seasonal sales fluctuations] with a slight upward trend over a two-year period. I’m asking for this because I am an L&D professional and I want to try to do the kind of analysis that my [sales-based] clients do. Be sure to make this data realistic, including successes and failures, and at least one disruption, such as a competitor launch or a sudden regulatory change.”

3. Iterate with genAI to get this right. The initial dataset will have problems, and you’ll have to hang in there to correct them.

4. Choose goals. Use genAI to reformulate the phase 0 goals in my actual assignment to match the synthetic data and the real reporting needs of the business area. Give it the data, the goals I supplied to my students, and ask for a comparable set of goals that involve analyzing the data. Be sure they include descriptive statistics and basic plotting.

5. Analyze, using code. If you don’t already know python, create python code that completes these goals. If you python, use a different language you don’t know.

6. Debrief and learn. You know how to do this better than almost anyone in your organization: get together with colleagues after you’ve tried the impossible. Share chat transcripts. Share stories. Share successes. Failures. Concerns. Hopes. Questions. Consider the implications. Plan.

Now, at a whole new level, you’re credible with genAI, data analytics, python, and - perhaps most importantly - your internal customers’ business. You’re probably pretty far ahead of most of them, at least on some of this. Good.

A universe of possibilities and imperatives is now open to you. You can use genAI for rapid generation of customized learning material at scale. Or create “intelligent” knowledge bases that learn through interaction with users and phase out static FAQs and job aids. The list goes on.

Step 1: Focus on the threat to informal skill development

But for now, I’m going to pivot to the silent skills killer that lurks in your organization, right now. This is the one I alluded to above - one that’s going to become dramatically more dangerous as your internal clients get their hands on productivity-enhancing generative AI and start implementing it around their organization.

Let’s say they’re motivated to explore genAI possibilities. So ideally they get a fresh set of information about what their workforce does. What their tasks are. What those cost, and what the ROI is. Maybe you’ve even helped them with this analysis. Then they select an area where they believe their workforce and KPIs could get a great boost by integrating generative AI into the work. And they go for it.

If you stayed in pre-2023 L&D mode, you’d mostly help them with the reskilling plan to facilitate this. Who needs what kinds of training, job aids, and learning infrastructure to set them up for success? How will you know when you’ve gotten there? You might also help a bit further upstream in job design. Then you’d implement to support, lather, rinse, repeat.

Here’s the trouble. They - and you, if you’re not careful - will be sacrificing on the job learning on the altar of that short-run productivity. The immediate first crop of users will - fates willing - get better results for themselves and the business, but they will do so in a more self-serve way that limits vicarious learning for other people in the firm. Junior members of their occupation. Nearby administrative assistants. Colleagues. Even folks more senior to them. Less participation in real work means less challenge, exposure to complexity, and human connection. I explore each in detail in my upcoming book, but they are the lifeblood of the kind of informal learning we take for granted - and that has underwritten our progress for thousands of years. This self-serve threat to the expert-novice connection was real before intelligent technologies, but it’s even more true now.

To step away from more obviously intelligent technologies - and perhaps closer to home for you - consider the work from home trend that emerged during the pandemic. People who know my work regularly message me, worried about their adult children’s skill development at work, for example. Here’s a typical one from Ezra Zuckerman, a Deputy Dean at the Sloan School of Management at MIT, publicly on twitter (notably *after* Covid):

Work from home was a safety issue at the height of the pandemic. Then it became a productivity opportunity. And when surveys show morale improvements with remote work, it’s hard to pin down reasons to bring people back together. Skill development is a critical one, but its signs are subtle, and the cost of inaction shows up years later in ways that are very hard to pin to their original causes.

L&D’s two moves: Skill + AI and Skill x AI

L&D needs to flip the script. It’s time to invest much more heavily in the informal learning to make up for the gaps in this age-old learning infrastructure. I detail some of the implications of this in my Harvard Business Review piece above, and go into much greater detail in the book.

Sometimes this will mean a “Skill + AI” scenario: L&D professionals can find ways to design, implement, or maintain technologies so that productivity and human ability are enhanced. The basic question here is “how can we get value from this technology and ensure that users build valuable skill as they deal with it?” A tall order. Not always possible. But we’re often not even trying. CEOs and managers, this is your chance to make your L&D folks your strategic partner. Considering a big new investment in technology? Bring your L&D leader to the table and let them ask this question of each vendor. Back them up. Ask vendors to do better. And when it’s time to implement, keep the question alive - for managers, for workers, and for your leadership.

But the deeper play is yours, too: Skill x AI. Your organization can race ahead - and the quality of work life can significantly improve - if you work towards a genAI-enabled network of human experts, novices, and AI, focused on building human and AI capability right in the middle of work. Many will push for a “one AI size fits all work” approach. You know better. My Step 0 assignment and your prior expertise put you in a fantastic position to assess which processes could be amplified or extended with these tools. This goes way beyond AI-assisted matches between experts and novices who happen not to work in the same physical space or organization. This is literally a new fabric for the expert-novice connection—where simply by engaging with it, both humans and AI learn faster than they could on their own, enhancing human relationships and our sense of fulfillment along the way. A lot of the tools to enable this are already on the table - in fact we have decades of pre-genAI research that shows automation can play a positive role in facilitating skill development.

It’s time to put them all to work in the smartest way possible to help our organizations adapt quickly and sustainably. The L&D function was made for this moment, and can be remembered as the internal leaders who rewrote the skill playbook just in time to avoid the skills iceberg above and capitalize on the incredible opportunities ahead.

So go ahead. Give yourself that Step 0 challenge. Connect with each other, and your leadership. And get to work. We need you more than ever.

Nearly four years ago - just after I gave my Ted talk - I began the slow, messy, ill-advised journey of writing a book. I may tell you a bit of the backstory later, but I mostly wanted to get you the insider scoop that it’s here!

You know my work, so the story will be familiar: we're threatening informal skill development by the way we're handling intelligent machines. But this is optional. Through vivid, real stories and the best available research, I show the hidden code that underwrites skill development, detail the trillion-dollar technothreat to skill, and offer a path forward that enhances both human and machine ability.

I’m so grateful for the early feedback I’ve received. It’s a bit overwhelming, actually:

“If you’re worried about your skills becoming obsolete, this book may be your saving grace. Matt Beane has spent his career studying how to gain and maintain expertise as technology evolves, and his analysis is both engrossing and edifying.”

Adam Grant, #1 New York Times bestselling author of HIDDEN POTENTIAL and THINK AGAIN, and host of the TED podcast WorkLife

If you want to learn more or care about this cause, I would deeply appreciate any and all support: 

1. Pre-order The Skill Code. The book will be available in hardcover, kindle, and audiobook formats. It goes on sale on June 11th.

2. Visit my new website for more insider information, including all the advance praise - from quite a stunning crew.

3. Forward this post to share the good news. If you’re excited about the book and the content I’m sharing here, my guess is your contacts will be too. Now’s your moment to brighten their day - and their horizons.

I so appreciate your feedback and interest. Congratulations on the choice to subscribe to my Substack, and welcome to your early bird seats on the book ride ahead!

With gratitude and delight,

Matt Beane

P.S.: as the book launch approaches, I’ll likely be writing for other outlets, so my posts here may not stick quite as tightly to the biweekly schedule I’ve been keeping. But my first-pass, early thinking will always live here.

“Once i got a grasp, it finally became a tool.”
- Master’s student, reflecting on a quarter of using genAI to build software

The world is obviously awash in commentary about the implications and nature of genAI. Will it kill jobs? Will it create them? Is it too biased? Is it less biased than we are? Did its creators have to steal intellectual property to train it? How fast is it improving? How should we regulate it? Is China going to win the AI arms race, or is the US? When is it good for it to be free, open software like Llama, when a paid, proprietary one like ChatGPT?

It’s critical that we ask these and many other questions, and that our global conversation reflect diverse perspectives. Generative AI is clearly one of the most consequential general-purpose technologies we’ve ever seen, and we need make sure we are informed so we can make wise decisions about it.

The more I’ve consumed the global conversation, the more I’ve concluded there’s a subtle bias in many accounts. One that really threatens our ability to get good data to answer our questions. It’s not about race. Gender. Socio economic status. West vs. east. Occupational differences. Political leanings. Silicon-valley bro-hood.

It’s about skill.

There’s a hump to get over with this technology. A journey of going from 0 to 1. A hard-to-articulate understanding of what it is and what it’s capable of - and a sense of awe (which includes fear) and possibility - that only comes from struggling with it towards something you previously thought of as impossible. Win, lose, or draw, at the end of this kind of work we become more capable, confident, and have something in our bones that helps us make more grounded guesses about what this technology means for us personally and for us as a species.

Most folks don’t go from 0 to 1. They try genAI out on something straightforward, get a bit of an understanding, then think they know about it. And for sure, they do. Knowing how to get a first draft of a convincing email, or an essay, or proofreading help, or explanation of a concept... that’s all important. Useful. But those kinds of tasks don’t really stretch the user or demonstrate the true power of the tool, so we walk away - at best - with a simplistic understanding of what we’ve got in our hands.

For instance, you probably know about genAI “hallucination” by now - the fact that these systems will produce very convincing false information when we ask a question. If you don’t already have some expertise in the domain you’re asking about, you can get hoodwinked. This can even happen if you know that these systems hallucinate! Of course many, many folks still don’t even know this basic fact. They are much more prone to taking genAI’s output as gospel, or waving LLMs off as hallucination machines.

But there’s a deeper layer of expertise about hallucination - one that many of us never build: a fine-grained, intuitive understanding of when and how genAI hallucinates, and what consequences that has for complex work relying on genAI. Sometimes, for instance, hallucination turns out to be very helpful feature - just on a personal level, it can help us check our own knowledge, provide creative ideas, or motivate us to learn more. To pick another example of this shallow-vs-deep skill problem: some of us might know by now that ChatGPT (and other LLM-enabled tech) will give you better results if you tell it to “take a deep breath”, or “think step by step.” Good. Important that some of us know this. But that’s not the same as understanding why and how that works - knowing how to program social machines. You might not be able to say what you know about prompting, but you know how to weave psychology and sociology into your prompts to get better results - and what bullshit on that topic sounds like.

There’s a whole line of research on this deeper kind of expertise - the kind that allows you to respond productively to surprises and uncertainty, the kind that helps you judge cause and effect in complex situations, the kind where the tool starts to feel like a part of you, but you get its limitations and possibilities in a way that’s hard to articulate with words. If you want to learn more here, you could do way worse for yourself than to read Doug Harper’s magical “Working Knowledge: Skill and Community in a Small Shop”. Doug was an ethnographer and a photographer, so the book’s filled with detailed, brilliant photos that show him apprenticing to Willy, a master blacksmith, mechanic, and tinkerer in an upstate NY repair shop.

Science on this like Doug’s book tells us: you’ll only get to deep skill with genAI by persisting with it on a complicated problem. Success and failure are secondary - you learn a lot by trying, figuring out what works, what doesn’t. What’s possible. And that’s a moving target, because the more you try, the better you get at it all.

New findings: learning to code in three weeks

The rest of this piece is more or less devoted to proving the above points by giving you a first-in-the-world peek at data from an educational experiment I ran in the fall with massive help from the amazing Brandon Lepine, a PhD student in my Technology Management department here at UCSB.

For the first phase of this project, we asked students to write working python code (aka create software) to analyze a complex dataset. We gave them three weeks (in the midst of a full curriculum of other coursework). And we told them to work alone. No help, no tutorial from us, no googling. Just you, ChatGPT or Bing, and the assignment. Figure it out.

Here’s the rub. They didn’t know how to code:

So out of a class of 26, 3 were professional software engineers, 3 could “do some okay stuff” with code, but 20 (77%) had no or next to no skill. In fact these numbers concealed a fair bit of terror at the prospect of this assignment. A significant chunk of the students who rated themselves as a “1” had deliberately avoided coding throughout their educational and work experience. Some hadn’t ever opened a terminal window on any computer, ever. Some didn’t even know what the word “Python” meant if it wasn’t an animal. They weren’t kidding with that “1”. From the point of view at the beginning of this piece, they were a hard 0.

We made them try anyway.

Students had to use genAI to write code that would analyze an open dataset about multiple construction projects (e.g., “Report the total number of open and closed tasks by task group”) and visualize that analysis (e.g., “Create a bar chart of the total number of overdue tasks by project.”). Alone.

This was a task that most of them would have thought was impossible, if they saw it before coming to our Master’s program. They wouldn’t have even considered trying. So this is totally unlike saving time on a writing task with a first draft, or doing some online research. That’s a task you know you could do, more or less well, and you turn to genAI for a productivity and/or quality boost. For many of my students - and most adults out there in the world - this task was the equivalent of brain surgery.

Here’s how we turned this wall into a hill: the bulk of their grade depended on the “quality” of their interactions with generative AI. I judged this by three criteria: candor (did they honestly and precisely say what they needed), grit (did they hang in there and keep dealing with problems until they were fixed), and creativity (did they find new ways to look at the problem that allowed for better results, did they ask clear, relevant, and useful questions). My working theory was if they were honest and specific about what they wanted, hung in there, and played around with some curiosity, they’d win. Yes, they’d get a correct answer on the assignment (almost everyone did), but more importantly they’d build their own skill and confidence and get a much richer, more “in the bones” sense of what generative AI could do for them.

Despite all this “help” (some felt like it wasn’t enough), students found this work difficult:

Once they got through that solo work, we put them in teams in phase two, and they had to write software that was genuinely usable by real technical project managers to solve a pressing problem in their everyday work. And they had to test it with real users, modify it to incorporate their feedback, then demonstrate their working system to the class.

Click here for the entire assignment - Phases 1 and 2, including the timeline, tasks, grading criteria. Please let me know if you use it, or if you have suggestions - I’d love to hear what you learn!

But the main point - and the thing that literally got my jaw to drop - was students’ own assessment of their comfort using Python to solve problems. I won’t spoonfeed this to you - take a look:

In the beginning of the quarter, most (18/26, or 69% - figures in the green bars) were “very uncomfortable” with Python. Again, this was a massive understatement for many. After just three weeks, most (again, 69%, figures in blue bars) were between “neutral” to “very comfortable” with Python. You could say they were lying, but their chat transcripts, work output, code told the tale: the average grade for Phase one was 95.7%. They’d gotten it. It might not be their go-to recreational activity on a Saturday evening, but they could make a digital machine to do complex work for themselves, where before they wouldn’t have even thought to try. Brandon and I were gobsmacked. We expected some improvement in comfort, but nothing like this.

I would have rated myself a “2” on this scale before the course began, by the way.

I took coding courses at MIT, and these relied on Python, but I hadn’t touched it since. I had worked around plenty of engineers at my startup and as I studied and collaborated with roboticists, so I knew about coding languages, embedded software and sensor fusion, understood basic principles of good software engineering, knew about the broader infrastructures required to create and run code (e.g., GitHub), had learned Mathematica a bit, too, but I couldn’t write you a useful webapp to save my life.

Then I did. In two days. With no help.

Here’s a tweet thread with the whole story, but in early September I realized I needed to overhaul the tool my Project Management students used to run monte carlo simulations (basically running a project plan hundreds or thousands of times with some random fluctuations in the length of each task). Instead of buffing it up in Excel again with the help of someone on Upwork - like I did every year, I decided to create a free, functioning, online app that did the same thing.

I knew ChatGPT was conversational. I knew it could code. I knew it *should* know about the work practices and tools associated with creating and posting a python-based app online. So I started.

Two ten-hour days later, I was done. The app was running. I was stupefied. Here’s the entire 219-page transcript of my interactions with ChatGPT, from noodling about the idea to deciding to try for it, to debugging (which I had to do offline, that wasn’t yet built into ChatGPT), to getting it posted on a service called Heroku.

About halfway through day two of this work I realized *this* was the new core of my fall course. Project management was the foil. If I could do this in two days, then students could do it in 10 weeks, especially if they collaborated. And they’d have a huge leg up on MBA-style students without this experience.

Roughly, that very much turned out to be true. But that doesn’t mean it was easy, or fun.

They struggled:

“It was a little difficult for me to understand how to set up Vs code and the GitHub account. so I had to go back and forth with GPT, as it doesn't really give you step-by-step details for a beginner to understand it right away… sometimes it used to be a little frustrating when I used to paste the error that I had in the code, and still again I wasn't able to sort the problem out. I had to really sort a few things yourself.”

Maybe the grading criteria nudged them in this direction, maybe they didn’t. But the students hung in there, asked for what they needed, and ultimately came out on top.

More to the point, they’d learned a *ton* about how to interact with these systems, what they were, when they made things harder, when they made things easier… and they’d been absolutely wowed at what they could do with this thing in their hands. Here are a few comments in that vein after the dust settled:

“Knowing that I can do things I never could before was extremely gratifying.”

“When you see your code, come to life and actually work, it is an amazing feeling. To see something turn in to a working calculator from code almost made me jump out of my chair with joy.”

“I found the use of Chat to build our tool to be a fantastic component of the project. We were able to focus much more on what we wanted to do rather than whether or not we could.”

“When I realise it has become a strength to create something that I envisioned, I just couldn’t stop doing it.”

That’s not to say it was all positive. But that’s the point, too. For some it helped them clarify their concerns about the technology, but these were also much better informed through practice:

“I don’t think this qualifies me to work at a software company, but I guess I could gaslight them into employing me.”

“You must have a baseline knowledge of what you're asking, because it may give you wrong answers without your knowledge. You must always fact-check and proofread the responses.”

It’s one thing to know these things in theory. It’s another to have tried, failed, tried again, failed again, troubleshot, given up in frustration, come back, expressed that frustration to the very system you’re using to do the work, get better results, check them yourself, then… finally, finally… run your code and see it produce professional-grade results. You know these lessons on a different - much deeper level.

When asked what a potential employer should know about their work in this project:

“I think they should know how fast we all picked up Python after using Chat GPT, and how we were all able to make PM software in that language within 10 weeks, which is crazy!”

“We made a functional and useful product by the end. From past experiences with group projects, the final deliverable is usually underwhelming, but in this instance it was exactly what I hoped.”

“That we did it in the time frame, which initially seemed nearly impossible.”

Not bad. Not bad at all.

We have a raft of improvement ideas for the experience - a true gift from an engaged group of students. I’m confident next year’s students will have a better learning experience as a result, and that they’ll bring even more relevant capability to their new employers or ventures after they leave our program. Perhaps most importantly, they’ll be in a great position to help their new colleagues, leaders, and customers make more informed decisions about what genAI is, what we can do with it, and what to watch out for. That’s probably far more valuable than any actual coding skill they have.

We have too much skill-free critique

It’s of course hard to verify, but my strong intuition is that many of those who issue harsh critiques about generative AI have not gone from 0 to 1. They haven’t put it to work on something they thought was impossible, tailored their genAI interactions to suit their needs, persisted through adversity, taken creative or even playful routes forward, and gotten a result. Nor have they then worked in a group of people to produce a collaborative outcome.

It’s obvious that going from 0 to 1 would not change everyone’s views on everything. That’s not the point. Many of us would clearly retain concerns about ethics, bias, intellectual property, profiteering, control, inequality, and even skill - and we need healthy critiques of generative AI to move forward wisely. But those critiques would be radically more informed about the practical, real possibilities, limitations, and threats associated with deep use of these tools - in a way that was tied to the critiquer’s sense of confidence and skill. Suggestions would be more nuanced. Limitations would be described more precisely, allowing for healthier fixes.

But maybe most importantly, going from 0 to 1 gives each of us a profound sense of what’s possible with this technology, and that we can turn it on a dizzying range of very complicated problems that before we would have treated as impossible.

One of those problems is genAI itself. We’ve essentially acquired the equivalent of undifferentiated electricity and sockets, delivered to billions of households in an instant. Of course that’s going to create massive problems. Just letting those run their course is totally unacceptable. But if each of us takes a little time and invests a bit of sweat equity, we can become conversant in this new mode of getting things done in the world, and we can help guide the global conversation more responsibly.

Want to get in on the action?

If you don’t know how to code, and especially if you’re fearful of it, do phase one yourself.

That’s right. Click on that assignment above. Read through the requirements of phase one. Follow them to the letter. Or if you want a different challenge - let’s say you know python - just modify the assignment to push you. Don’t know C++? Java? Make yourself do it there. Don’t want to analyze construction data? Find some data that mean something to you. Or create your own data.

The main thing is: make sure the task feels nearly impossible, not just difficult. And don’t let yourself get help, aside from either Bing chat in creative mode or ChatGPT (with GPT-4 enabled). Stay signed in so you can save your transcripts. Send them along, if you’d like - heck, I shared mine!

Be candid. Gritty. Creative. No matter the outcomes, you’ll get results you didn’t think possible, and you’ll become a skill-informed global citizen at a time when we need to be making better decisions than ever.

Yes, really. Here’s a brief rundown on key events in just the last week (each link has at least one video and plenty of supporting material):

• On January 3rd, we learned that a large group of researchers collaborating across places like MIT and Google announced Mobile Aloha, a two-arm robotic rig for just over $31,000 in parts. And it’s run on open-source software that learns when humans control it remotely (known as “teleop”). Per the authors, it contains *four* breakthroughs. It: “1. Moves fast. Similar to human walking of 1.42m/s. 2. Stable. Manipulate heavy pots, a vacuum, etc. 3. Whole-body. All dofs teleoperated simultaneously [me: multiple joints can move at the same time]. 4. Untethered. Onboard power and compute.” The key here is that the mobile aloha system could perform complex tasks autonomously with only 50 training examples - even when researchers tried to mess with it by moving objects, closing drawers, and putting chairs in the way.

• On January 4th, researchers at Google DeepMind announced SARA-RT: a system that makes transformer-based models more efficient on a robot, and allows users to direct robots through natural language. Basically, they had many robots crawl various facilities. They’d take in image data, use an LLM to describe each scene (e.g., “a table with some fruit and stains on it”), use an LLM to identify potential tasks for the robot (e.g., “put the fruit in the basket”), check which of those tasks the robot’s body could do (e.g., no folding towels because the robot only has one arm), then pick which task to do based on what was least familiar to the entire fleet of robots. That way, each new task contributed maximally to the knowledge of the central system, making all the robots more capable, sooner - with lower compute cost.

• On January 6th we were told that with ten hours of training (we’re left to wonder what that means, exactly), Figure’s humanoid robot learned to make k-cup style coffee at normal human speed in the lab, including recovering in creative ways to spontaneous, minor physical mishaps. Truly amazing. Even if the team had been working at this since the CEO’s Dec 26th announcement about the unexpected, transformative power of foundation models for robotics, it’s still amazing. Oh, and this should break a few stereotypes about silicon-valley engineers insisting on bougie, Kyoto-style coffee (ngl, that’s my go-to).

• “Oh yeah?” says 1x Robotics on January 8th, when they posted a demo video of their humanoid robot pouring and serving coffee to passersby in a public space. I haven’t seen a demo-off like this since perhaps the Darpa robotics challenge from 2013-2015.

So, what’s going on here? Why does are the LLM and robotics streams are merging at (sometimes literal) lightning speed?

The main reason has to do with what we saw from OpenAI in the late fall: multimodality. Over the course of the year, freely available generative AI went from handling only text to being able to receive images and produce images, and relate these to text as well. So, for instance, you could give ChatGPT a photo and it would tell you what’s in the scene. Here’s a fun example where Ethan Mollick asks the system to identify a very well camouflaged snow leopard in a distant, grainy photo.

Video is “just” a sequence of many images. And cameras and data storage are inexpensive, which means robots take in lots of video. So it’s much easier now for a robot (running an LLM) to immediately identify what’s going on in a scene, identify potential useful actions to take, and direct its control systems to take those actions. Given that these models run in near real-time, this LLM-enabled control loop includes correcting for surprises or errors as the action unfolds. Here MIT Technology review explores how this is being applied in driverless vehicles - which, like our dishwashers, are 🎶robots in disguise🎶.

Before LLMs, parsing what was going on in a scene, identifying potential goals, and taking action that’s responsive to dynamic conditions… these were herculean tasks, subject to incredibly arcane, brittle, and (therefore) specialized software that managed the problem differently. Tabletop physics models were popular. Neurologically-inspired approaches to computation were on the rise (they’re still quite hot). But with LLMs many more roboticists can get in on the action without hyper-specialized, ultra-rare skill.

If you want a deep dive on the robotics*GPT events in 2023, I recommend starting with these three world-class summaries by recognized leaders in this space, and following your nose from there:

• Karol Hausman (Alphabet/Deepmind)

• Jim Fan (NVidia)

• Brett Adcock (Figure)

But, in sum: deployable robotics is just barely springing out of the gate in 2024 with generative AI under the hood, but the progress is rapid, and resources are flooding in.

So what? We’ve been undercounting on AI*Jobs

In addition to rapid, under-the-hood progress in robotics, the last 365 days have seen a lot of analysis and hot air about the implications of generative AI for jobs and work.

In mid-2023, Rob Seamans, an economist at New York University, and colleagues published an analysis of the potential impact of AI like ChatGPT on all known jobs, as registered in a US-government-curated database on work called O*NET. O*NET covers the work activities for 1,016 occupations, breaking these down into 19,265 tasks, like “immunize a patient” or “operate welding equipment.” And I’ve mentioned before that, Daniel Rock, an economist at UPenn’s Wharton School, and colleagues from OpenAI published similar analysis. Both papers aimed to show how much each and every job was “exposed” to the automation that a general-purpose technology like GPT-4 represents. Exposure here means how many of the fine-grained tasks in a job could get a 50 percent productivity boost if the worker used GPT-style technology.

Many folks anchored on to the fact that a very few jobs that were at or very near 100 percent exposed—like blockchain engineer and mathematician. We’re drawn to black-and-white, “total” style change, for one. But also, it struck closer to home for the intelligentsia: it looked pretty clear that it was mostly white-collar workers who were most exposed to a new automating technology. Despite what some immediately claimed, this didn’t mean those jobs were going away—it meant that someone in those jobs would have to change the way they did almost everything, if they used GPT-style technology. That would require almost complete reskilling. But those jobs are rare. The jaw-dropping scope of this reskilling problem becomes apparent when you read the conclusions in these papers that are perhaps less flashy. Try this one on for size: 80 percent of all working adults have jobs that are 10 percent exposed to GPT-style technology. In the US, that’s 108 million people, and around the world, that’s 2.7 billion people who might need to relearn 10 percent of their job.

Now consider this: all that billions-scale exposure and potential change is just from the “mind” side of the human skills ledger. Those two papers only considered tasks that were subject to automation by software that deals with ideas, writing, images, and thought.

But now you know: we’re also making rapid progress on the “muscles” side of things. In fact, the story of 2024/5 is that progress on the minds side is accelerating progress on the muscles side.

Erik Brynjolfsson, Daniel Rock, Tom Mitchell, other colleagues and I are working on a rubric to estimate the exposure of physical tasks to these developing technologies. Preliminary results make it clear that many, many physical tasks in the O*NET database will be exposed to profit generating, robotic automation, especially as LLMs facilitate their development and deployment. Millions more people will be exposed, and in many cases, they’ll be in more physical jobs that would not have been all that affected by intangible, GPT-style automation. So, it turns out that a 10 percent job change for 2.7 billion of us is probably a significant underestimate. More soon as we analyze the data.

There are at least two differences between robots and servers-and-screens AI that should stop us from leaping too quickly in this direction, however.

Robots are not bicycles

Steve Jobs famously thought of computers as bicycles for the mind. Tools that we build that dramatically amplify our capabilities so when we use them, so we outperform any naturally evolved being on a task (in the case of the bike, energy efficient transport per calorie).

Currently, this is the way generative AI has effects in the world: as force multipliers when a human puts them to use. Think back on those GPT papers above. When the authors are referring to “exposure”, they are talking about tasks where the user picks up some generative AI and uses it to do a task they previously did without the tool. They can often do it faster, better, or both. They might even replace someone who isn’t using the tool, or their organization might not hire more people (thus eliminating potential jobs) by mandating that employees use the tools. But there’s typically no productivity gain unless a human attempts to perform a task with this new “intelligent” tool in their hands. You can gut-check this against your own reality: when was the last time ChatGPT did something without a prompt? Never is the correct answer. Once you prompt it, sure, it does stuff.

Robots are, roughly speaking, the opposite.

A robot is a physical device that can sense its environment, use that data to build a plan for how to act in the physical world, then take that action - and the cycle repeats. You give the robot a set of objectives, define operating parameters (e.g., safety, timing, pace), set it up and let it run. Robotic vacuums are a great example - once they’re set up, they just do the vacuuming. They don’t amplify your personal vacuuming actions. They cut you out of the equation entirely. This is the same with robots in assembly, materials transport, inspection, pick and pack, and yes, even washing dishes. Most robots are like this.

There are notable exceptions: the da Vinci surgical system (the robot I studied for a few years) is notoriously referred to as a “master-slave” system. It takes no direct surgical action without a surgeon pushing or pulling on its controller. We also have powered exoskeletons that give a boost to workers trying to lift or carry something, walk long distances with lots of weight on their back, or for physical rehabilitation. No worker action, no robot action. For wondrous, cutting-edge work in this territory, go look into research on collaborative robotics: software and systems designed explicitly for dynamic physical collaboration on uncertain tasks between humans and robots. Julie Shah’s a recognized leader in this area, and her lab at MIT produces and tests amazing possibilities with corporate partners like Honda.

Let me stress that this is all optional. Robots are not somehow necessarily more autonomous than software. We could have a world where robots primarily amplify human effort or serve as dynamic collaborators. Many power tools now fall in this category. We’ve just built a world where anything we might readily identify as a robot is far more likely to be self-contained, highly autonomous devices.

They offshoot of this difference is that - in most cases - when a task is “exposed” to LLM-enabled robotics, it will be automated completely. Amplifying or collaborative applications are quite rare. So the effect of automation via robot - say in terms of job counts, job composition, and reskilling - will be quite different than those associated with screens-and-servers generative AI. A good rule of thumb is probably: robots mean more replacement, less augmentation. This seems to spell more “all or none” exposure, rather than the “matter of degree” exposure associated with intangible automation.

So there’s your first difference. The second has to do with the fact that generative AI is made of bits, and robots are made of atoms.

Hardware is hard

Before you conclude that the all-or-none exposure insight means LLM-empowered robots will double the amount and rate of job change, take a deep breath and think through what it takes to make something like the robots we see in the news these days.

Or you might instead ask ChatGPT to take a deep breath (as I wrote before, this is one of a bevy of “social programming” techniques for improving output quality) and explain why commercializing robots is slow, compared to software. That’s what I did, and ChatGPT did a fantastic job:

Beyond the reasons above, I’d add two: first, the current and foreseeable cost of a robot’s physical parts and their physical capabilities means that robots are wildly expensive and inefficient way of automating something compared to software. The cost to produce that Mobile Aloha robot above was about $30,000. That’s not including the labor to assemble and test it. And it can only take about one useful action every five to twenty seconds. Sometimes an action takes a minute. And how long would you have to wait for the components to arrive? Oh, you wouldn’t mind if I threw a pandemic or trade war in there to snarl your supply chain a bit, would you? And what if I asked you for a million units?

Second, building and deploying robots means collaborating across many more disciplinary boundaries than in software. Mechanical engineers. Electrical engineers. Embedded systems engineers. Straight-ahead software engineers. I/O psychologists and/or Human Factors folks. Safety and legal professionals. Test engineers. And that’s just inside the traditional design and build team. The research on occupations and professions is crystal clear here (this now-classic paper by Beth Bechky does a great job also summarizing the research): cross-occupational collaboration is harder than intra-occupational collaboration. The more methods, beliefs, values, tools, and practices you have at the table, the harder it is to harmonize them. Software teams are more homogenous in these senses, so the work is easier. Faster.

Improving on any of this takes a massive thicket of interdependent technical and social innovation across numerous disciplinary boundaries. For instance just on the technical side of the ledger, getting a robot to run longer without external power means some combination of advances in areas like battery technology, weight of key materials (in the 2000s we got powdered metal, for example, which reduced the weight of key components while maintaining strength) and power efficiency (here we recently got 3d printed magnets, which can allow for more elegant, energy-sipping actuators - devices that make robots move). All that is slow, hard, and way more expensive per automated action than in software.

Because software is information made animate via controlled energy and we have an internet and distributed computers in place, you can ship and use software at almost zero marginal cost (a term economists use to refer to the cost associated with producing all units of a good beyond the first). Modifying and maintaining it can also be done centrally at relatively low cost, and you can push updates out at zero marginal cost too. It’s all… really easy and fast compared to robotics.

These two factors - robots as replacement and as hardware - mean that it’s not appropriate to directly extrapolate our predictions about the automating impact of generative AI into the robotics space. The implications will take some working out.

But the reality is that generative AI has breathed new life into robotics, and we’re seeing progress on really hard problems to do with the brains side of the robotics ledger. Orienting to an environment. Making sense of it. Planning potential actions. Deciding amongst them. Reacting to surprises. Even shifting goals or trading off amongst them. These hard problems are now easier, for a larger pool of roboticists. And at the same time foundation models are accelerating the basic science underlying many technologies, so the rate of progress on things like power, materials, sensors and even supply chains is likely to take a tick upwards. Making better robot bodies will get easier and easier because of generative AI. So it’s a safe bet that our work is significantly more exposed to robotics in a post-foundation model world than it was beforehand.

The key with this second insight, however, is that it dominates the first one. As much as some top-flight roboticists like to claim, the hard problem in robotics is getting 50,000, 500,000, and then 5,000,000 robots built and shipped to customers that have paid for them. Setting up those supply chains, ensuring six-sigma reliability and safety of those systems… well it’s the equivalent of starting the automotive or home appliance industries all over again. And no matter how inexpensive those robots become, they will never cost $0. Or anywhere close.

I made this point in 2013, and it stands: robots are rounding error in the economy in some senses. They’re really expensive, complex, and difficult to change, compared to software. Both technologies transform and transport things - it’s just that software does it to bits, and robots do it to atoms. Atoms are massive by comparison (sorrynotsorry).

One robot to rule them all

Here’s the potential phase change associated with all this - one that’s worth watching closely: multi-purpose robots.

Until now, most commercially deployed robots have been built to perform one task (e.g., a hulking arm for lifting car chassis around on an automotive production line) or a very tightly bound range of tasks (e.g., a robot that can pull different kinds of cargo through spaces also traversed by humans).

The problem with this 1:1 relationship is that any learning gains associated with that robotic system are hard to port to other robotic chassis or applications. Some of the hardware might be useful in other contexts, but you’d still have to piece together a new rig from a blank sheet of paper. And some of the sensing and control software would probably be useful, but would require extensive tailoring given a new robotic chassis. So the robotics community has sensibly turned to a modular “library” of components - digital and physical - that can be repurposed and recombined without too much hassle. But even all this means that - by and in large - every new robotics project requires a from-scratch design, build, test, order, assemble, maintain work stream that’s quite idiosyncratic to the specific application at hand.

This is part of why investors, roboticists, and some industrialists have been pushing for some time for multi-purpose robots. These are systems that can handle a *very* wide range of tasks with only software changes. The humanoid is the most popular flavor of this kind of system these days. We know very well that the human form can handle a lot of different tasks, so it’s a tested template for a robot that can do the same. A humanoid robot was once a pipe dream, but is undeniably here, now: surf here for my up-to-date list of firms *commercializing* humanoid robots - not just building one that works some of the time.

There’s nothing magical about a bipedal human form for this goal. One of those robots has a wheeled base. Not all of them have five-fingered hands. In fact one of my favorite designs for a multi-purpose robot was CHIMP: Carnegie Mellon’s entry into Darpa’s Robotics Challenge. There’s never been a creature - real or imagined - that quite looked like it, but it could handle a wide range of tasks, some better than a human body could.

But the point is that if we create a standard for a multi-function robotic chassis (or even a few of them), then we could both manufacture and revise it at scale, and most robotic innovation could turn towards improving the software. And software learning gains from one context - how to wrap a gift, sew clothing, assemble electronics, make guacamole, or rivet an aircraft fuselage - could improve these systems’ performance across most other task domains. Then robots could become - as D. Scott Phoenix, the former CEO of Vicarious Robotics repeatedly said - “as ubiquitous and inexpensive as smartphones.”

A lot of diverse parties - entrepreneurs, venture capitalists, government agencies, engineers, scientists - are pushing hard towards this multi-purpose chassis target. Have been for over a decade, actually. But as evidenced by the explosion in my humanoids robot list - and related articles in the tech media - expect there to be a surge of effort in this domain this year and next: everyone recognizes that there’s huge, new value to be unlocked if we can solve this “one robot, one task” problem.

Such consolidation still seems like it’s years away. Say 3-5, at a bare minimum. But as it occurs, our physical work might become exposed to LLMs in ways that beggar the imagination, compared to today.

This doesn’t necessarily mean we’re all out of work. In fact it probably doesn’t mean that, if history and social science offer any predictive value. But it does mean that many more than 2.7 billion of us will have to change significantly more than ten percent of how we do our jobs. Which means that, even more than we might have thought, our short- and mid-term challenges are not job loss, but learning to do our jobs differently.

This is where the multiple roads of automation lead. As far as I can see and the data show, skill development is one of our critical challenges right now, and we’ve got to give it everything we’ve got. Head back to my first Substack post to reorient yourself on that problem if you like, but this is where I’ll focus my posts - perhaps for a while.

Most voices out there make it sound like the world of AI-change is a runaway train. Just about a year ago we got GPT-4 via a text-based interface. Things have exploded since then. We’ve gotten the ability to interact with these systems via images, audio, and video. In the last week or so we got adaptations that can create short, high-ish quality videos and songs in mere moments. We’ve gotten the ability to ask for code-based solutions to our problems - with these systems testing and debugging their own code. And now we can - just using natural language - program up tailored “GPTs” to solve focused problems (like improving our prompts, tutoring for HBS-style case writeups, and teaching us to identify rhetorical questions) that we can share with each other. Last week a startup (supported by the artist Grimes) announced it has embodied these kinds of agents into “Grok”, a plush toy that will interact verbally with children.

And this isn’t all contained in a few private, closed companies like OpenAI and Google: just in the last week, we’ve had huge new drops of open forms of generative AI that anyone can run on their computers - or even phones - for the cost of the electricity involved. They’re about as good as the GPT-3.5 system that ran under OpenAI’s ChatGPT just under a year ago. Thousands - perhaps tens of thousands - of top coders in the open software community are feverishly at work trying to surpass companies’ closed models. Many say they will achieve all this, and soon. On top of that, the open community is making it easy to get groups of GPTs to work together in digital organizations to solve complex problems - with only initial human guidance. Companies are now offering this functionality in beta trials.

Head spinning with “progress” yet? This advancement isn’t just good for run-of-the-mill administrative, writing, and analytic tasks for the masses: it’s enabling breakthrough science that likely wasn’t possible without generative AI. Also just in the last week we found out that - with a little artful scaffolding - genAI has produced genuinely creative breakthroughs in math and in biology: a new solution to an age-old calculation problem and an entirely new structural class of antibiotics - the last one took 38 years to find. Even Terrence Tao, the world’s top mathematician, has said he’s now better at math with these tools.

And yet…

Automation is slow

Here’s the thing about getting real world results - at scale - from new automating technologies: it’s slow and difficult, even when we want it.

In fact I talked a bit about this two posts ago, to help explain why - for most folks - there’s no need to rush on genAI. The decades-long delay between the arrival of new automating tech and widespread results from automation boils down to how organizations work, but I didn’t explain how that is, exactly. In fact, the most you’ll usually get on this point is some hand-wavy mention that it takes a long time to figure out the right way to put new automating technologies to use, and that lots of other complementary things have to change - in our lives or an organization - for that to happen.

But while it’s accurate to say “automation is hard and complicated, and therefore slow”, it’s also deeply misleading. Stopping there obscures what social scientists call mechanisms - the specific interaction patterns that actually cause outcomes to happen or not to happen. If we don’t try to understand those, we’re ignoring huge swathes of psychological and sociological research on how individuals, groups, and organizations change as new forms of automation emerge. If we do understand mechanisms, we can make better predictions about when automating something will be quick, when it will be slow, when it will be easy, when it will be hard.

One key reason automation is slow is because we don’t have the skill to put it to good use. And this even affects individuals. Let me share one hot-off-the-presses working paper on Kenyan entrepreneurs to illustrate. Productive automation using ChatGPT should be quick and easy for these highly motivated folks, right? They’re certainly not burdened by bureaucracy.

Wrong, it turns out. Or at least only partially right.

In this fantastically well designed study, Nick Otis (a PhD student!) and coauthors gave one group of entrepreneurs access to a ChatGPT-style business mentor to aid in their work. A control group got no such help. I highly recommend this detailed thread explaining the work. But in short, all of these entrepreneurs were super motivated to do well. They were taking significant risk to try to found a business, and rapidly tried to take advantage of the genAI they were given. They could automate instantly.

Some did race ahead, it turned out: the entrepreneurs who were already performing very well saw a 20% boost in key outcomes. But lower performers saw a 10% decrement in their outcomes. That’s right. They did worse.

And we get some insight into mechanisms here: it looks like it’s because “both groups sought the AI mentor’s advice, but that low performers did worse because they sought help on much more challenging business tasks.” In other words, even if you want to automate something badly and no one is there to try to slow you down, you can fail: either you don’t have enough expertise in the task, it’s highly ambiguous, risky, or all of the above. Trust genAI too much in those circumstances and you're liable to get hurt - not just fail to automate.

But let’s put fast-moving, untethered, hyper-motivated entrepreneurs to the side for the moment. The huge mass of the consequential automation in the world arises through organizations, and…

Organizations are still using dot matrix printers

Over the holidays, my wife and I traveled east to spend time with family (I’m in the air on the way home as I finish this). And I noticed: every gate had a dot matrix printer - a technology commercialized in 1968 (seriously, geek or not, go watch that explainer video). Compared to modern printers, these are more expensive to operate, harder to resupply and maintain, are loud, produce lower quality output, demand additional training, and require weird software and paper that are wildly out of step with modern corporate IT and supply chains.

On the surface, this is profoundly irrational.

But retaining archaic automating technology turns out to be a side-effect of the way organizations automate in general. Every organization, everywhere, has the moral equivalent of dot matrix printers in service - I bet you are already thinking of one or two examples in your context. And the same mechanisms that drive the dot matrix phenomenon also impede automation that relies on the latest technology.

Anyone who’s yelling loudly about the need to put LLMs to use right away doesn’t understand this reality. And if you’re a pragmatist - someone who wants to get great results with this new general-purpose technology without sacrificing your organization or team to do it - you need to understand this reality.

In this section, I’ll explain more why that is. In the last section, I’ll suggest some ways we will use genAI itself to change this game in a way it’s never been changed before.

Organizations are like houses

I’ll illustrate the status quo through two papers of mine. The first is coauthored with Paul Leonardi this year, and it’s summed up in this figure:

The short story? For new tools like ChatGPT to add business value, they need to be integrated with the foundational elements of an organization: its structure (e.g., roles, reporting relationships, norms), infrastructure (e.g., functions like recruiting, procurement), strategy (i.e., what it’s trying to accomplish), and field (e.g., industry). Paul and I show this by reconsidering nearly a hundred years worth of study of organizations, work, and technology, and suggest that it all indicates one thing, summed up by this quote from Stewart Brand in his magnificent “How Buildings Learn”:

“Fast and small instructs slow and big by accrued innovation and by occasional revolution. Slow and big controls small and fast by constraint and constancy. Fast gets all our attention and slow has all the power.”

Some parts of organizations change more slowly, because they are more foundational, and these are interdependent with the parts of the organization that change with the whims of global fashion. From this point of view, ChatGPT is the new Slack - a cool new automation tool that a lot of individuals are jumping on. Sometimes this can become the tail that wags the dog - a lot of individual Slack users within a company can drive the stewards of the company’s IT infrastructure to implement Slack for the firm. But most of the time, new tools like Slack will not get put to use in the organization for a long time - or maybe ever - because no one does the work needed to mesh them with the firm’s infrastructure, structure, and possibly strategy. That’s a huge reason why Microsoft’s teams - a demonstrably crappier product - has wildly outperformed Slack in the market. It’s compatible with the IT backbone of the organization, which was developed, installed and maintained via Microsoft-connected product and knowhow.

On top of this:

Organizations are like investment portfolios

Another key point to consider is that keeping an organization running takes resources - cash, people, space, time, attention, effort, and so on. And while investing these resources produces resources in return - you get things like customers, revenue, employees, skill - no organization has unlimited resources.

That means organizations invest unevenly across their automation portfolios. Some automation gets more attention, cash, and people than other automation. And we’ve learned that organizations overwhelmingly prioritize investments in proven, familiar domains - because it’s the equivalent of printing money.

I showed the implications of this portfolio problem this year in a paper examining robotic surgery (open access). To get my data I spent over two years at one hospital that disrupted its automation portfolio by buying an upgraded surgical robot while retaining the older one.

To maximize short-term ROI and expand into new markets, the hospital shunted the best supplies and infrastructure to their new system - and gave privileged access to inexperienced talent. That safer, easier environment let newer talent get up to speed faster. But that also meant the hospital preferentially allocated its most experienced talent to their older surgical system, while depriving that robot of preferred maintenance and infrastructure. It broke down, often in the middle of surgery. Surgeons had to operate with one eye blinded. Nurses had to doublecheck “failure proof” components. Scrubs had to stash extra supplies and force various components open and closed. Sounds like a recipe for inefficiency and harm, right? It did to me. So I collected years of surgical outcomes data to check. It turned out patients did just as well on what became known as the “shitty” robot.

Read the paper for the (literal) gory detail, but there was no dip in surgical outcomes because that experienced talent invented workarounds to produce the same results as those achieved with their newer system.

Here’s the twist, and the link to the slow automation story above: getting good at those workarounds meant that top surgical talent only “whined” about the older robot to management. They never got data about how difficult it was to use, and how it was breaking down. And management never came by to check things out themselves. Ultimately, that limited execs ability to consider new automation options. From their point of view, things were working just fine. So the hospital was running to stand still because it was carrying aging technology that was difficult to use, put extra burden on top talent, and that was degrading over time, given reduced maintenance.

Thinking dot matrix printers yet? You’re on the right track.

This portfolio-driven blindness to the pain and waste of the automation status quo is part of what will block many organizations from noticing and investing appropriately in generative AI - the most consequential new technology to hit the global market in a couple decades. They’ll have a thicket of automation that seems to do the job just fine, and keeping it all working may mean they may not even see the genAI opportunity in front of them. If they do, it might not look that attractive, because their current ways of automating things are familiar, reliable, and effective. Yet those ways will have accreted slowly, with some (usually older) parts of the portfolio getting short shrift, and therefore demanding more and more hidden effort to keep them on their feet.

In a way, this is all a recipe for what Bob Sutton and Huggy Rao call “friction” in their new book. The longer organizations run, the more they accumulate processes, rules, even culture that make it harder and harder to take on something interesting and new. Automation - essentially a blend of work processes, tools, skills, beliefs and assumptions - is no exception. Organizations accumulate and then master various forms of automation which facilitate good performance at a certain level in certain external conditions. But when a potentially disruptive tool comes around, it can only become part of the way the organization automates things if most of the people involved in related structure, infrastructure, and firm strategy make significant change that has implications for many, many other processes, tools, and situations. When the new automating tool is also a general purpose technology - a tool that helps in tasks throughout the economy - the range and intensity of this change explodes.

So: most organizations won’t try to automate anything with genAI, at least at first.

Many will ban it. Not because of the risk to IP, or for ethical reasons, but because it’s incompatible with the foundational elements of the firm, and its portfolio of prior automation investments.

Experiments will be small.

And all of this will frequently be Just Fine. Those who prefer rapid change, exploration, and risk may dunk on fuddy duddy organizations that are taking it slow with genAI, but betting the farm on a new technology is usually not prudent. The friction and inertia associated with the foundational elements of an organization are part of what allow that organization to adapt to change. If a few small experiments yield solid results, bigger ones will follow, enabled by the very foundational processes that impede rapid, wholesale change. Over the course of a few years to a decade, a firm can reconstitute itself to take wholesale advantage of a new general purpose technology - without tearing itself apart.

And yet…

Change is the new Stasis

Take a look at that timeline.

For most of human history, the average organization experienced *zero* new general purpose technologies. From about 1250 to 1750, we got one about every 150 years. So still, practically zero. Since 1950, we - and our organizations - have experienced five. And three of those have come just since 2000.

Call it exponential if you want. ChatGPT did: it quickly fit the data with a four term quadratic polynomial. It’s clearly not a linear trend. And our most recent arrivals are self-improving in unprecedented ways. So from this point of view there’s a new intensity layered in there on top of a simple increase in the rate of accumulation. (It’s important to note, by the way, that some argue that our best inventions are behind us - I highly recommend that argument if you haven’t heard it before)

Previous ways of running a firm and “ingesting” new automation may not cut it anymore. Top business schools teach techniques for generating “best in class” versions of that house diagram in my paper with Paul. Those techniques were devised between 1950 and 2010 or so. They don’t presume an exponentially increasing rate of change in the arrival of general purpose technologies. And they certainly don’t presume generative AI - most notably its likely accelerating effect on a great deal of innovative activity.

I don’t have a well worked out alternative to offer you. No one does. That’s partly because discovering it will require doing it. The first organizations that operate in ways more amenable to change in general - and adopting “intelligent” automation in particular - will likely race ahead, and others will learn from their example. Andy McAfee’s latest book “The Geek Way” - the definitive, theoretically-grounded account of what makes silicon valley organizations tick - likely offers some clues here.

But even the best of pre-GPT organizational wisdom isn’t liable to explain how organizations will handle automation moving forward. First off, you can see above that putting new automation to work creates unintended side effects for an organization. It can divert resources away from other efforts, which can create strain and inefficiency. But also adaptive behavior that blinds everyone to opportunity. No one pays or can even see these costs clearly. These dynamics are akin to climate change, given rapid growth in our civilization’s energy consumption. For that we’ll probably create…

AutoMGT

To help our organizations keep pace with exponential change, we share management responsibility with intelligent machines.

Right now, humans are in primary control of each of the elements of an organization - whether lightweight and rapidly changing, like tools, or massive and slow, like infrastructure. When a new form of automation comes along, it’s the human interpretation and action that controls how an organization assimilates it. For a new tool to become part of the automating fabric of the organization, many dispersed humans with diverse interests, skills, roles, and information need to change the way they think and act - even if it’s just a little bit.

That worked pretty well from say 2,900bc to 2022.

If we keep creating general purpose technologies at an exponentially increasing rate, we and our organizations will find ways to adapt at that rate too. Humans can’t do that without help. The only way to keep pace with genAI-enabled automation will be to automate the automation process with genAI.

We’ve been scaffolding ourselves towards a hybrid human-machine form of management at least since the advent of the spreadsheet, the algorithm, and the sensor: managers now get massive, rich datasets to inform decision making, and these are automatically analyzed and presented in ways that help those managers make higher quality decisions far faster. A manager in a top performing organization today can help their organization adapt to change far better than they could in 1980.

What we are just barely starting on is getting LLM-generated analysis and advice on how to structure our organizations in response to change. We can now use generative AI to analyze the vast repository of data that our organizations collect and produce to propose productive changes to tools, structure, infrastructure, symbols, and strategy. Once we build genAI-enabled systems like this, humans will still decide whether and how to buy new technology, change the org chart, rebrand the company, or change its strategy. But excellent managers will rely on this technology for decision support on these topics. For an early example, check out WorkHelix - cofounded by three colleagues - Erik Brynjolfsson, Andy McAfee, Daniel Rock (coauthor of the GPTs are GPTs paper) - and James Millin.

This is AutoMGT. The equivalent of a software-generated design feedback for a semiconductor engineer, predictive analytics for a leader on climate change, a legal and safety compliance alert for an architect designing a neonatal intensive care unit. It’ll just be about how your organization is built and how it runs. If this reality unfolds, we will get used to systems like this. At least in “situation normal” conditions, we will learn to accept their recommendations as routinely as we do the terms and conditions for our phone software updates. We’ll scan, assess, then click “yes”. It’s not the same as changing the organization itself, but this is literally what happens now for front-line hiring and layoffs in the warehousing organizations my team and I have studied for the last few years. A human manager makes the call, but software provides the targets. We’re already in the habit.

We don’t have good precedent for understanding what it will be like to work in an organization reliant on AutoMGT. What organizations like this will be capable of. And what the unintended side effects will be. That last point is key: AutoMGT-style organizations won’t be unambiguously better. If the sociology of work, technology, and organizations teaches us anything, it’s that some of those side effects will be nasty, accumulate slowly, and will be practically invisible to everyone involved - until it’s too late. But the adaptive benefits will be proximate, clear, and significant, so we will take the deal.

Living and working in organizations will start to get a bit… weird as we co-design organizations with genAI. Exciting new opportunities, subtle deep threats. Now is a time to pay fierce attention to both sides of the ledger. Study, test, and share findings on how to handle this new territory, and do our best to make choices that insist on improved productivity and enhanced human welfare.

With genAI - ChatGPT, Bing, or Google’s hot-off-the-presses Gemini - all of us have new ways to do our work faster. Better. You probably know this firsthand and from people you know. A letter. A job application. Some homework. All done in minutes, not hours. And if you doubt the anecdotal, you can turn to a growing crop of rigorous studies (here’s my favorite recent example, known as the “Jagged Frontier” paper) showing these effects - improved productivity with essentially zero training. The raw interface is intuitive, thanks to over two decades of global smartphone use: we just chat with these technologies via text, upload documents and images, and we get back high quality text, code, and images in return. It’s really quite something: even after hundreds of hours of use, the results still inspire awe for me.

So in a very real way, we’re more skilled. With genAI in our hands, we have the ability to get higher quality results more reliably and quickly, compared to those without it. What else is expertise, other than the practical ability to solve a problem?

A lot, it turns out. And the “other” part of expertise is under threat.

Expertise - like most human phenomena - is social, not just practical. It boils down to trust: our belief that someone claiming to be an expert is credible and capable. Callen Anthony and I discuss these two aspects of expertise in our recent paper. The problem we found in our data is that senior members of an occupation struggle with building skill with new technologies, because appearing as if they are novices is problematic. People - both the public and junior members of the occupation - need to believe that someone knows what’s going on and how to handle it.

It’s not good enough for us to improve our ability to resolve problems, and for others to do the same. We need to find a way to trust that people have skill before we see it in action. Otherwise experts won’t get the opportunity to work in the first place, and the system breaks down.

Expertise is in “genAI check”

I’ve mentioned before that using genAI threatens skill in one subtle way. You can limit fruitful struggle, creative detours, and interaction with others, which in turn can limit your skills growth. We don’t have to settle for that. In that post I give some tactical guidance to not only avoid the brAIn drAIn, but to get the opposite result - better skills than you could get without AI.

Here I’m focused on the credibility threat to expertise, and will land in a comparable place. Yes, there’s a way to go from being in genAI check on your skill to the opposite: using genAI itself to present our expertise in a new way that enhances people’s trust.

But let’s start with genAI check, and begin on the practical side, closest to “actual” skill.

When you’re hiring someone… when you collaborate… when you join a team… these are all moments when you have to decide: are they good enough?

Even if you’re hiring a contractor to solve a specific, apparently obvious problem, you’re coming to them because you’re not sure about the best way to interpret and act in the situation. You know your gas oven intermittently makes a poof noise, seems to go out and then sounds like it restarts itself, but you don’t know that for sure. Or perhaps you have some extra cash and are looking to invest it for the long haul. You hear that an index fund is smart for a nonprofessional investor, but you don’t know if that’s true in your case, and if so, which one to choose.

All you know is something’s wrong, or there’s an opportunity in front of you.

Research on expertise makes this clear: when you need to trust someone’s expertise, you’re concerned with more than whether they can handle a particular challenging task well. You’re not sure what the task is! You need to trust them to assess a complex situation well, choose an effective method to address it, and then reliably execute on that method. And this is a far bigger deal if you’re going to have multiple interactions, or the problem is really ambiguous or dynamic. You’ll need them to repeat this process for the same problem multiple times, for new problems that are different. And as long as we’re talking “real world” here, there are other pressures that require trust. Sometimes a supposed expert will have ample time to work, sometimes they’ll need to respond quickly. Sometimes they’ll get second chances, sometimes a single screw up means a catastrophe. Sometimes we’ll get rich data on their work because we’re in person, but often these days we’ll be remote, and just have to... that’s right: trust them.

And if we don’t trust them, we won’t hire them. So they won’t get a chance to prove their expertise (and thus build it, but that’s another story).

Now let’s take a second look at the leading study I mentioned above. It’s very well done by top researchers, and is the first to assess worker performance across a range of potential tasks for someone in that role. In this case we’re talking consultants dealing with various ideation and writing tasks for things like marketing and business strategy. The world and the authors often hold it up as reinforcing the idea that LLMs are a powerful boost to individual performance. Not false. And sometimes we equate that with skill. The consultants who had less skill to begin with see the biggest performance gains using the technology. That’s a real and important finding, obviously.

But its title - “Navigating the Jagged Frontier” - and findings actually highlight the trust problem I’m on about, almost more than they do the productivity boost. That’s because boosts were uneven, and weird. In some cases, users’ performance got worse than it would have been without the tech. In other cases, neutral. The study shows that genAI is an uncertain and powerful tool. On top of that, we do not need to reveal the fact that we’re using genAI to produce our output. From homework to job applications to memos and business proposals, we now know that a huge majority (64%, says the latest high quality survey!) of users don’t disclose they are using this uncertain and powerful tool.

That’s… not exactly the recipe for trust in expertise. In fact, it’s a threat.

Why? We know that experts might be using genAI to help them interpret and intervene in challenges. In fact any time we’re counting on someone for their expertise and they don’t say they’re using genAI, we have to reasonably assume they are, and are willing to pass that work off as their own. This goes way beyond consulting, now that these systems run on our phones and can interpret and produce images. Your plumber or computer repair tech could pull out their phone in the middle of a job. And while that may often be okay, we know the effects are semirandom. We know about hallucination. And, really, what happens if their cell isn’t getting signal in the basement?

At least for now, to place our faith in expertise, we need to trust the human behind the machine.

Getting out of AI check

The generic recipe for getting out of AI check is simple: genAI transparency.

Let’s frame this in terms of your expertise.

These days if you want to earn someone’s trust to perform some work, it’s incumbent on you to assure them that you’ve got a handle on this AI thing. That you know your stuff, can get great output from genAI, can tell when it’s wrong, and can do solid work on problems where it’s not helpful or available. And you need to occasionally allow someone to independently audit your work so they can see how “brittle” your expertise is - were you just coughing up what the AI gave you? Were you asking it for a second opinion? For search? Are your prompts suitably sophisticated? The list goes on, and inquiring (client) minds want to know.

This is literally what genAI-engaged professors like me do now with our students. Sure, you can use genAI, we say. We know you’re going to anyway, and we can’t detect it. But the new standard is: show your work by including your full genAI chat transcript with your finished product. If you don’t share a transcript, you’re either really good or you’re lying. And you’ll have to stand and deliver live in class anyway, so we’ll find out. While we’re trying to develop expertise, not rely on it, the need for trust and resulting push for transparency is very similar.

We live in a genAI-turbocharged world, now, and this is the standard that you’re going to need to meet to earn trust in your expertise. Practice AI transparency.

But how to do this, exactly? Just posting your ChatGPT logs online is one way to do it, but ultimately this isn’t very user friendly. Simply saying you use the technology is great, but that leaves too many open questions about your ability to handle unusual or dynamic problems. Potential consumers of expertise need an easy way to assess what you can do now with AI, and how you’ve worked to preserve a suitably broad and flexible range of abilities needed for the messy real world. We need new ways to present our expertise to earn that trust.

One step we can all take: make a genAI portfolio

genAI is new. Unrefined, in many ways. It’s still early days. And it has far reaching implications for work across most occupations. That means it’s not an exaggeration to say that perhaps a hundred million working adults are in AI check - right now. That’s a mind boggling array of tasks, situations, tools, and expertise. What’s an efficient way for anyone, in any kind of work, to give some assurance to potential consumers that they’re handling genAI well?

How can we demonstrate genAI transparency?

I’ve got one practical answer for you, and it was inspired by Alexis Chew, an undergraduate student in my “Managing Technology Organizations” course this quarter.

As I wrapped up my second class on generative AI (early in the quarter, because I was going to mandate students use it), Alexis approached me and asked a simple question: “how do we prove our genAI skills to potential employers?” She thought they might value the skills she was building in my class, but that they wouldn’t ask and she didn’t know how to tell.

I said I had no idea, but that it was a great question. I turned it around to Bluesky (the other blue-icon-themed social network), and Jesse Shore dropped in one word: “portfolio”. This seemed instantly on point: the portfolio is flexible genre that is designed to demonstrate expertise for diverse audiences. And Alexis was intrigued - she has diverse skill sets - so we converted a healthy chunk of her course grade into a focused project to produce two things: a research-backed template for a genAI portfolio, and a reasonably fleshed out portfolio for her own expertise.

The first of these documents outlines the range of portfolio types out there, and examines how they work. By accounting for the moving parts inside each type, Alexis was able to propose a few strategies that would be particularly effective for genAI transparency. It turns out, for instance, that “case study” and “narrative” logics seem particularly helpful. All humans understand stories, and cause-and-effect relationships. If we tell stories of a problem we faced, how specifically we used genAI to address them, share the output and its impact, we’re well on the way to helping people understand our hybrid human-genAI expertise and our interests. Alexis, for example, is focused on environmental science and art, and the case studies in her genAI portfolio show how she used this new technology to learn more and get better results in both domains. And in case you’re wondering, yes, she used Bing extensively as she produced both these documents, and submitted her chat transcripts along with them.

She and I are excited to share both, here. Please bear in mind that we view these as works in progress - we agreed it was better to get the ideas out there soon rather than to polish them up for final presentation:

genAI portfolio toolkit (by Alexis Chew)

Alexis Chew’s genAI portfolio

We hope you’ll adapt, share and use these, and that they inspire many to explore portfolios as a way to account for their expertise with this new set of tools. This may be especially important when we’re dealing with the many folks who are not clear on what the technology is and what it can (and can’t do). And a genAI portfolio should be valuable far beyond the leaving-undergrad-and-seeking-a-first-job category. Been learning genAI and looking for a mid career switch? A genAI portfolio could help underwrite the move. Have a profile on a gig platform like Upwork? This might help you stand out there, too. It can also work as a screening tool for you on the consuming side. If you’re choosing whether or not to join a work group or organization, hire a contractor, or new employee, you can ask: do they have a portfolio of genAI efforts to share?

Trust is a must

No matter how skillful you are, if people don’t trust you, you won’t get to put your expertise to work. We have seen the arrival of many, many productivity enhancing tools before now - from fire to the potter’s wheel to steam engines to calculators to computers and the internet. None have been quite so broadly capable or “creative” as genAI, while also delivering strangely inconsistent results. And none have been quite so easy to conceal during use. The threats to faith in expertise are unprecedented.

If genAI is useful to you now - or you could imagine it helping you get better results in the future - then people will have new, significant questions about your expertise. Many millions of us are in the same boat. Whether it’s via portfolios or other forms of AI transparency, we will all need to address these gaps in confidence to get permission to put our expertise to work.

It’s moving FAST, and no one knows what’s next. Sam Altman’s out at OpenAI! Wait, no, now he’s at Microsoft! Wait, no, he’s back! Is all this because they had achieved AGI (artificial general intelligence, aka the holy grail) and the board thought Sam wasn’t being safe enough with it? Or is it because he was going too commercial? Does it matter? And that’s to say nothing of Google/Alphabet’s looming announcement about their ChatGPT buster, Gemini (coming Q1 2024, we’re now told). Microsoft is spending nearly 14% of its total company capital expenditures on AI, apparently twice as much as Google and Meta, and nearly four times as much as IBM and Amazon Web Services (AWS). And finally, without ever using the words “generative” and “AI” in the same sentence, Apple is reportedly investing a billion dollars a year in generative AI to preserve its position.

And non-tech organizations are following suit. Price Waterhouse Cooper has announced a billion dollar investment in generative AI technology, saying it will transform its organization - and workers’ jobs - as a result. Visa has announced it will invest $100m in generative AI possibilities for financial transactions. Khan Academy is well into its deployment of KhanMigo, a supportive chatbot for millions of students and teachers with generative AI under the hood. This may run fast, far, deep, and wide: if some commentators are to be believed, every healthy organization with even a modicum of data will spin up its own “enterprise LLM” to gain competitive advantage. And for now, governments are pretty quiet about any limits to this techsplosion.

There’s sound, there’s fury, and… there’s blistering progress on a new computational paradigm that is almost alien compared to what we’ve all become accustomed to.

Where does that leave us?

You might be thinking: When do I use search, and when do I do prompting? And what IS a good prompt anyway? How do I write one? Shouldn’t I be learning to code with ChatGPT by my side? Or programming up one of these new GPTs to… do things for… someone? And, sorry, hang on, what even is a GPT again? And where do I do all this - is it ChatGPT, Bing, or what are these AI-enabled search results I see in Google now? Is that the same? I’m not spending hours a day on this stuff, and when I try it it just kind of is… okay? Sometimes even a waste of time. Am I doing something wrong? Am I going to get left behind?

Yes, you will be left behind

From one point of view, a small number of people and organizations are racing ahead with generative AI, and most of us are stuck in the past. If you don’t jump, you lose!

To see this dynamic we need reach no further than the hot-off-the-presses working paper from a star-studded team at Harvard, Wharton, MIT and Boston Consulting Group. The authors and the news media make a lot of the “twist” in this paper: consultants with lower skill get a bigger performance boost than those who already had high skill. That’s an interesting and important finding, but focusing there has meant we often breeze right by the jaw-dropper - here, straight from Ethan Mollick’s substack: “Consultants using AI finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without.”

We have decades of research showing similar dynamics - for individual and organizational productivity. Clark Gilbert’s research showed this for the advent of the internet and the news industry. Most players froze, or moved slowly. Those who pounced on the internet leapt ahead. Don Sull showed us that when radial tires came along, Firestone Tire & Rubber responded by accelerating activities that contributed to its prior success. A few hospitals in my nationwide research of robotic surgery did the same: they refused to use robots and doubled down on “open” and “lap”(aroscopic) methods. And recently, we’ve been getting studies about the deployment of robots, showing that the firms that invest early get significant productivity gains, hire more people, and their competitors (who don’t buy many - or any - robots) lose marketshare and shed jobs.

The key link between organizational failures and your skills and job is sitting right there in those studies. If you’re not in an adopting firm, you don’t get to use the new tech, so you won’t get healthy opportunities to build skill with it.

We can extrapolate this to the atomized world of work today: all of us have always-on access to the same, top-of-the-line automating technology. Every freelancer, knowledge worker, and student has a strong productivity incentive to use it. Many of us will try, and some of us will race ahead with it. This is what Erik Brynjolfsson means when he says “workers who work with generative AI will replace those who don’t.”

In case you’re now firmly convinced that you’re in trouble - that doing your job as you always have means someone out there gets these kinds of immediate boosts from using genAI - I have great news.

There is a giant hole in this argument, wedged open by decades of research. One you can coast right through without lifting a finger. You might even end up ahead as a result.

Most early adopters are going to pay… dearly

The truth is that most of those who try to race ahead fall flat on their faces. We don’t study them much. We focus on winners.

A classic, clean example of this comes through “The Computer and the Dynamo”, a marvelous study of the adoption of the dynamo by economic historian Paul David. The dynamo - a revolutionary, general purpose power source compared to steam - became commercially available in 1880. Yet David reminded us that manufacturing productivity only took a notable bump by about 1920. The point David makes in the paper is that improvements come very slowly we need to change organizations - work processes and infrastructure - to take advantage of new general purpose technologies. And that kind of change takes a lot longer than just buying the new technology.

But without intending to, David makes an even deeper point. One you can get at just by asking what was happening with dynamos in industry from 1880 to 1915 or so.

It wasn’t nothing. Early adopter businesses everywhere were trying to integrate dynamos into their operations in one way or another. They failed. For at least 20 years - and more likely 30 - they wasted money, interrupted operations, and most likely caused economic, emotional, and physical harm to their workers. A primary failed approach, David shows us, was to replace their steam engines with dynamos. The problem there was that most factories had a single, giant steam engine in the middle of the building. This engine turned a giant shaft, which had numerous belts attached to it. Those belts distributed power to equipment throughout the building:

Firms just swapped out the steam engine for a giant dynamo. They kept the belts in place, because replacing those would be very expensive. But also because they couldn’t imagine another way of handling the power distribution problem. The ultimate solution was to put many smaller dynamos throughout the building, which allowed for more flexible manufacturing operations. But the deeper point here is that getting there meant many factories and firms burned significant capital, time, and employee welfare trying this technology out - and most of them failed.

History doesn’t record these failures. All that waste and harm. The huge cost to most who made early bets on a new general purpose technology. How many people died, were injured, or just plain lost their life savings by being first to deal with the dynamo in their workplaces - say in the first five or ten years it was available? We don’t know. How many factories suffered similar fates? How many businesses paid a bigger price than their gains - lost employees, slowed production, or went out of businesses entirely? Again, we haven’t really asked. But the lack of productivity gains proves the case: industry did not have major success through the technology before about 1910. Only a very few lucky firms would have achieved that success much earlier. That’s 30 years of near-guaranteed waste and failure - an entire adult’s working career!

We could ask similar questions about all general purpose technologies through history, from fire and the wheel through electricity, the internal combustion engine, telephony, the computer, the internet, and now, generative AI. The pattern in studies of technologies like these is clear: going early is risky, hard, and nobody really wins (or, more factually, lots and lots of firms and individuals lose) until we figure out the right ways to design and use them.

This is why - if you just take a cold, unemotional look at the facts - following is the best move. Unless you are simply compelled to try the latest new thing, you should do a little low-risk, high-potential-payoff experimentation with unalloyed generative AI, and to wait for the risk takers to figure things out. You’ll be fine.

The apps are coming, the apps are coming

The other reason why I’m confident you can wait to use generative AI is that we’re already seeing some green shoots when it comes to its utility for everyday people and organizations. The apps are coming.

Think of generative AI as raw electrical power. An extremely powerful resource that can be used for lots of things. OpenAI’s big innovation was to package it up in a chat interface. That instantly made it more useful for billions of us. But that’s basically like getting electricity to your home. Fine, you have electricity, outlets, and electric motors on hand, but what can you do with them?

Most of us shouldn’t start messing around with motors, wire, and whatever we’d hook it all up to. We’d get electrocuted or maimed, or at best waste a ton of time. What most of us should do is wait for entrepreneurs to make reliable appliances. Lamps. Table saws. Dishwashers. Standup mixers. Heaters. Refrigerators. Water pumps. Then we can get to work buying those and putting them into our homes.

Appliances are specific, physical tools that make electricity and motors - both general purpose technologies - easy to use. When we get these for digital general purpose technologies like the internet or the smartphone, we call them applications, or apps for short. The first iPhone came with only a few apps - Jobs famously showed it as being “an iPod, a phone, and an internet communications device”. Then, eventually, they opened it up to developers who made thousands of apps, some of which made these devices far more useful.

Many firms are doing this right now for generative AI. You can bet your future on it. OpenAI and Google can’t do this alone. They need the world to work out how to make generative AI useful. So if it’s not your strong suit already - if you don’t already have a *very* clear line of sight to something innovative - your best bet is to stay a little connected, do some low-cost learning and experimentation on occasion, and wait until someone hands you something genuinely useful.

Microsoft’s new “Copilot” in their office suite provides a nice, easy example to think with.

Browse through that page. You’ll see: OpenAI’s LLM functionality has been woven into each of Microsoft’s office apps. Outlook for email, Word for documents, Excel for quantitative data, PowerPoint, and so on. In each, you get context-aware, personalizable tips, guidance, and automation that matches pretty well with your prior experience, goals, and skills. Their systems won’t just chat with you. With your permission, they’ll read your emails, prior writing, and maybe even consume your job description to give you help that’s tailored to your specific work. It can even “coach” you to improve over time - which has the potential to deal with the huge AI-threat to your skills. Using Copilot is kind of like going from a hand saw to a circular saw, powered by electricity. Slightly different game, same task. And the electricity/genAI is in the background, powering it all. You’re not dealing with it in it’s raw form. It’s appified.

Another example that’s a bit more out there comes from a startup called Lindy.ai. They’ve leapfrogged ahead of OpenAI to create a system that lets you spin up groups of AI agents that can independently cooperate with each other to handle complex projects. You create “Lindies” for each task syou care about - handling customer support, entering data, doing market research, taking notes, scheduling appointments - then you “onboard” them with a bit of text explaining their job, you connect them to your apps (like slack, google drive, calendar, docs, linkedin, zoom, github, and so on), and then they get to work solving problems for you.

OpenAI took a babystep in this direction a couple weeks back when they launched “GPTs” (agents you could program to be good at one thing - I made one to help with Harvard-style case writeups and one to help you write better prompts) but they’re not connectable to other apps, and can’t interact with each other. Small firms have huge incentives to unlock useful apps like Lindies, and we can count on them to do it, or die trying (many of them will).

This is what’s coming. For most anything you’ve used computers for before - from creating text and images, to sending and receiving messages, to browsing content, to video calling… whatever it is, it will soon be transformed into apps that are readily and reliably useable to you with very little retraining cost.

The lessons of history and your personal preference here are probably aligned: you shouldn’t go first, or early with unrefined general purpose technologies, generative AI included. Those who do have a good chance of hurting themselves pretty badly, and more commonly just waste a bunch of time and money.

Just take a deep breath, go about your business, and a new layer of helpful “intelligence” will be built into many of your day to day work tasks, probably sooner than you think.

ps: this is my fourth Substack post.

Many of you (over 250 in just under a month!) have been here from the beginning, and your positive feedback has inspired me to continue, though I will take a brief break over the holidays. Thank you!

I have a favor to ask. If you’ve found my writing valuable, please share it on social media, with colleagues, friends, and family. I’d greatly appreciate it - I only write about things that I think the world genuinely needs to hear right now, and I can only get these messages out there with your help. And by the way I’ll be sharing some exciting news with you all soon - before the rest of the world gets to hear it!

“Take a deep breath.”
“Give it a shot.”
“Please try hard, this matters to me. I believe in you!”

Before 2022, these statements weren’t computer code - now they are.

You read that right. What was once an exhortation to a fellow human is now an intelligible instruction for a machine. And research is starting to show that instructions like these make a difference in the output you get - we’ve seen significant performance improvements and decreases. Computing and automation will never be the same, but nobody quite understands how or why. It’s time to get smart about programming social machines.

Programming used to be hard

To see this new programming picture, you need to understand the old one.

Before 2022, if you wanted to get a computer to do something for you - from scratch - you had to write precise, procedural instructions for it in notation that wasn’t readily interpretable by novices. This notation allowed for a host of machine-executed operations on well defined inputs and outputs: repeat iterations of a task, based on the last task output (known as recursion), performing a task as long as certain precise conditions were met (“for” loops and if/then conditionals, for example), taking in input in certain ranges, storing data, doing math, producing graphic output… the list goes on. This notation is difficult for many to learn, but underwrote modern society:

We call this “code”.

It executes exactly as it is written, every time. That’s the point. Those who can write code can make digital machines that create value for many others, because they can get a copy and run it themselves. That’s what I did with the code above: I made a webapp that runs a statistical simulation of a project’s duration. Anyone with the url can access and run it for free. But a great deal of software isn’t free, and that puts a lot of power in the hands of the few who learned this notational form (and way of thinking, really).

That’s all changed.

Now programming is easy

You’ve probably heard by now that we’re in an era of coding “in natural language”. Practically, this means you can issue instructions to a machine in English (the lingua franca of the world, for good and ill), and machines will fulfill those instructions. And your instructions can be fiendishly complex, containing conditional logic, recursion, and so on - just like code. And you can even ask genAI to produce traditional code as part of this kind of programming. For over a decade, investors and technologists have anticipated and invested in this possibility, partially because one of the key bottlenecks for software production was how hard it was to write code. The idea was that allowing people to code in natural language would dramatically expand the global software engineering population.

And it’s starting to happen. Anyone who can think procedurally, define a goal, and engage in chat-based dialog can code up a digital machine to get work done. OpenAI recognizes this - their big announcement this week was a tool to allow us to spin up these machines (which they call “GPT”s) much more readily, and to share them (for profit) with others via a marketplace. Ethan Mollick explores how to use them - and their implications - here.

So, fine, we’re all software engineers now, and getting to work is as simple as knowing your ABCs. The barriers are down.

But…

Now, programming is also social

But the barriers are also different. Higher in other ways. Weirder, but also familiar.

That’s because they’re social.

At least since an anonymous google engineer declared their AI to be “sentient”, we’ve had the intuitive sense that there are… additional features to the form of notation we use to program generative AI. genAI systems like ChatGPT produce and respond to queues that are not directly related to the task, or its goals. These are queues like the ones at the opening of this article, related to the mindset of the agent doing the work, and their relationship with other agents. Humans work like this, and we’ve invented two branches of social science to study both aspects: psychology and sociology.

Psychological programming

Right now, the social programming out there is psychologically oriented - these are prompts that include things we might say to shape someone’s thoughts, feelings, perceptions, or effort.

Psychological prompting was in part inspired by early efforts to probe the “mind” of large language models (most think these models are not conscious, but this has not been proven). Creative psychologists like Michal Kozinski at Stanford started to subject these models to basic tests, showing they exhibited behavior consistent with a “theory of mind” (i.e., an understanding that they’re interacting with other agents with private beliefs and values). We’ve since gotten papers showing that ChatGPT has personality - in other words that it gets coherent scores on well validated personality tests.

But this approach didn’t emerge after hearing from psychologists via papers. Nonpsychologists have been on to this possibility for quite a while longer. You’ve likely heard one of the first successful attempts in this domain: “think step by step”. This prompt was uncovered in May of 2022 by researchers at the University of Tokyo and Google Research as they tried to get GPT-3 to perform better. It was terrible at math, and word problems in particular. The results were astounding: for instance just including this in a prompt to solve a problem in MultiArith (a standardized math assessment) improved model performance from 17.7% to 78.7%! In September of 2023 researchers at Google’s Deep Mind offered an improvement: “take a deep breath and think step by step”. This odd combination of emotional and cognitive information offered improvements of up to 50% on Big-Bench Hard tasks. This is a suite of 23 tasks that humans find very difficult, covering a wide range of topics beyond arithmetic reasoning, including symbolic manipulation and commonsense reasoning.

Despite early results, there are clearly huge untapped and unknown potential here. In part that’s because in most cases this psychological programming is not designed by psychologists who could draw on theories about human psychology to pick powerful prompts. Engineers and computer scientists just kind of messed around and found out!

A wonderful recent example that bucks this trend - written up by a group of coauthors from academia and Microsoft - named an entire domain of psychological programming: emotional prompts. It turns out that indicating that you’re feeling strong emotion (hope, fear, anticipation) or increasing the intensity of your expectations (e.g., “please try hard”, or “you better be sure”) has dramatic and generally positive effects on genAI performance. These kinds of prompts got 115% gains on Big-Bench Hard tasks - more than double the gains from the previous attempts! The key difference in this study is authors drew on psychological theory to both explain why this might work, and to choose amongst potential prompts. This makes it much easier for others to expand on these programming techniques.

With this approach in hand, we can make much more rapid, efficient progress in psychological programming. Here are just a few exciting bodies of theory that explore different ways to improve human performance. As far as I know these have not yet been tested on LLMs:

• Instilling a growth mindset (cultivating the mindset that you can improve)

• Creating psychological safety (reducing the risk of speaking up)

• Nudging towards “fast” and “slow” thinking ( intuition vs. rationality)

• Encouraging ownership

• Expressing or producing gratitude, wonder, or awe

…the list goes on. Seriously: psychologists and computer scientists should be getting together to save time, improve results, and build new theory about psychological programming. LLMs won’t respond to these prompts exactly like humans do - they’re not human, after all.

But when it comes to programming social machines, psychology is only half the game.

Sociological programming

As far as I can tell, we literally haven’t begun to experiment in the sociological direction. As the name of the discipline implies, this has to do with the social conditions associated with our thoughts, feelings, and action: are we a subordinate or a superior? Higher or lower status? Aware we’re being observed or not? In a hierarchical organization or a “flat” one? A bureaucratic organizational culture or an entrepreneurial one? Collaborating with others in real time or working asynchronously? Humans behave in predictable ways in certain social situations, and often this is a far better predictor of our behavior than any psychological differences between us. Analyzing the effect of social conditions on human behavior is the domain of Sociology (and Social Psychology - the field that sits between these two poles that initially attracted me to social science).

To start concretely are a few examples of potential sociological prompts that should have an effect on LLM performance:

• “You are a high-ranking executive responding to a junior employee.”

• “Respond as if you worked in a traditional, rule-bound organization, then as if you worked in a dynamic start-up.”

• “Imagine responding anonymously online.”

• “As a member of the open software community, address this issue.”

The first prompt draws on role theory, which suggests that we behave according to the social roles we occupy. The second draws on organizational sociology: we know that organizational culture shapes human behavior in areas such as formality, rule following and creativity. The third draws on research on the online disinhibition effect - in which some people self-disclose or act out more than they would in person. And the fourth draws on Social Identity Theory, which predicts that our responses very significantly depending on the norms, knowledge, and status base of the group represented. As with psychology, the list goes on, and on, and on. We’ll have to prioritize.

Quick note: we might not have seen sociological programming yet because it doesn’t work for some reason. That would be huge, and very interesting. LLMs encode psychological dynamics but not sociological ones? Why? That would be another good reason to publish failed prompts. But it’s also possible that they do work, but we haven’t tried them (in a research way). That’s because they’re just not as intuitive, or attractive. Humans have a much harder time understanding sociological forces, or believing they’re real when they do. We prize free will, and understand ourselves as contained beings: we don’t like the idea that our situation exerts more control over our intentions and actions than we do. Either way, we’ve got to look into this.

Another reason we need to look into sociological programming is we’re on the brink of introducing tools to the public that rely on multiple LLM agents interacting with each other. For almost ten months now (an eternity in this genAI thing) - software engineers everywhere have been spinning up “multi-agent systems” to solve really complex problems. Think of a system that helps you spin up a small, “just add GPT” organization - complete with different roles and responsibilities - and you’ve got the picture. These have names like AutoGen, MetaGPT, Langroid, and AgentVerse, and they’re amongst the most popular shared software projects on the planet. We just haven’t seen them commercially launched yet.

In the 90s, Cliff Nass (a researcher at Stanford who’s since deceased - a tragic loss) gave us the theory of Computers As Social Actors (or CASA). But this idea - and the research tradition it inspired - was centered on the notion that humans treated computers as if they were social actors. Not that the computers would respond differently as a result.

Now they do.

Learning to program social machines… socially

Our first task is to understand exactly how this is so. Especially given all the different LLMs out there, and the different guardrails those companies are putting on these LLMs, it’s not obvious that all the findings of psychology and sociology will neatly predict these systems’ reactions to a very broad range of social programming.

The next task is to figure out what works, and why. Finding useful tactics is helpful, but if we don’t have principles to help us understand what’s going on, our progress is going to be far slower than it should be. We need to develop theories that predict what works - in the words of one of Kurt Lewin (one of my intellectual heroes): “Nothing is as practical as a good theory.” Knowing why something’s true saves time and effort.

Academics or researchers in industry, if you’re reading this and agree, please forward this post to those you think might run experiments to test these things out. This is a target-rich environment, and only a few of us are trying. Happy also to talk about collaborating in this space.

And if you’re just out there in the wild with ChatGPT, I have a challenge for you, and a request: try this stuff out - especially sociological programming - and let us know what you find in the comments. I’ll do the same. This is an exciting time where you have exactly the same, state-of-the-art LLM in your hands as any researcher does. As I do generally in my research, we’re all on a hunt for a positive needle in a negative haystack. The next big discovery in social programming could just as well come from you as from some credentialed academic.

The key is we all have to share what we’re learning. We have to do this socially. Right now the incentives for isolated use of LLMs are strong. You get a quick result and you move on. That blocks a key mechanism for progress: learning from all our individual experimentation out there.

So let’s start programming these social machines, socially - as in sharing, connecting, and learning together, as we learn individually. See you out there!

Hundreds of millions of us use genAI.

One of our most common tasks? Writing resumes, cover letters and applications to go after jobs. Some of us are even using it to respond to live questions in remote interviews. And we all know many students use it for homework - writing papers, essays, research reports and so on. Some teachers are grading that submitted work with genAI, by the way, and building tools to help others do the same. Other tasks are more mundane - we use it to write an email, to think through a recipe, to do travel planning, and so on. And far beyond the mundane, very high skill workers are a year or more ahead of us, applying specialized, often pre-ChatGPT genAI in their tasks. Software developers have had a genAI assist at least since GitHub copilot was introduced in 2021. Professional writers, lawyers, and consultants have been using these tools to create, edit, and summarize text. And scientists across numerous disciplines have been using it to find new research questions and directions, design new instruments, collect data, and write up papers.

And it’s working: the immediate results are striking. (There’s a hidden cost, too - I’ll get to that in the next section).

Some waste effort and rework aside, resumes are getting us jobs. Homework is getting better grades. Emails are more persuasive and clear. And not for specious reasons. The materials are better than they would be otherwise. I’ve told my students that I’m expecting better writing from them, now that these tools are here, and they’re delivering. And we can do more of all of this, faster. On the professional side, Software developers are writing better code, more than twice as fast - while feeling more satisfied and focused. Writers get their work done 40% faster and 18% better. Consultants are getting improved results, too. As for scientists? For them it’s letting them “revolutionize the entire data science pipeline”, propose new protein structures, and optimize magnetic containment of superheated plasma for next-gen fusion power.

Finally, society-level productivity gains from genAI should be significant, because we use it a lot: in the last month, chat.openai.com and bing.com (another instance of GPT-4 hosted by Microsoft) had 2.7 billion visits combined. Reuter’s recent poll indicates that 28% of working Americans - that’s just over 4.5 million people - use the technology “regularly” at work, despite the fact that 10% of employers ban its use outright. And while we don’t have good data on this yet, it seems clear that productivity gains will emerge because folks understand that this is erratic technology. Most of us get better results with these systems because we know they “hallucinate” regularly, that you need to check their work, and that new versions, plugins, and competitors are arriving regularly. Beyond current stats, all of this activity - from use across skill levels to sheer usage volume - is increasing and expanding. We’re still finding out ways to put this general purpose technology to use.

So, to sum up: hundreds of millions of us are regularly using generative AI and getting more, better results sooner. Great, right?

Subscribe now

Be careful what you wish for…

It can be. Let me stress this: it will sometimes be wonderful to boost quality, volume, and ease of work for the 2.6 billion working adults who can apply this technology in ten percent of their jobs. In the grand sweep of history, this can give us the productivity boost we’ve been missing, creating new value for everyone.

But there is always a tradeoff, and here, it’s about deskilling.

Since the 18th century, we’ve understood that we sometimes lose skill - and the advantages that flow from it - when new automating technologies arrive. Part of the reason for this is routinization: to take get the best results from a new technology, we simplify the work involved. Invented a conveyor system for parcels? Good, now the human job of moving parcels doesn’t require walking around or dealing with other people. Invented a calculator, or an electronic spreadsheet? Great, now the human just enters the data and hits enter. No arithmetic or cross-checking required. Throughput and quality go up, but that new work system demands less of the humans involved.

To illustrate: here’s a quick vignette from Jax (not his real name), a 25-year automation veteran in a leading AI-enabled robotics startup in silicon valley - recorded during my team’s three-year, nationwide study of advanced automation in warehousing:

“So let’s say an injection molding system requires three operators to run at its manual capacity. When they shift it to an automated status, now it's two operators. In that three person configuration, there was probably actually a fair amount of skilled labor that was involved. The local supervisor knows damn well that that job before took a fair amount of savvy - sort of like, ‘come on Bessie now, let's get this part [produced].’ And now it is just push button.”

Working with the new technology makes these workers more productive - higher quality parts, faster. And it’s less demanding - not as dangerous, not as stressful, or difficult.

And it’s deskilling. Pouring plastic beads into a hopper, pushing buttons, and inspecting parts takes less ability, so the longer a worker does this new job, the less capable they become. They literally lose skill. New workers in this job simply don’t get a chance to build it. And often, deskilling drags down job quality with it, including pay and dignity. This is the well-accepted finding going back to the 18th century, stretching from Smith to Marx, Braverman to Burawoy, Fernandez to Vallas, right up to today.

Many will counter this argument by saying that rational employers will reallocate humans to tasks that do require (and therefore develop) human judgment and skill, because that’s where human capability is most valuable. And so the net effect of automation is good for the average individual, organization, and certainly society.

This is misleading in two ways in general, and one way that’s unique to genAI.

First, you can’t move everyone to a new, better job that requires and develops valuable skill. A good number of humans have to do the new tasks in the midst of the newly automated process. You leave two operators on the injection molding machine. Instead of two paralegals reviewing contracts, you have one with an AI assist. Instead of two surgeons to do an operation, you’ve got one - and according to many of them across 18 of the top teaching hospitals in the US, compared to open surgery, robotics is like “bumper bowling” - a little too easy for comfort. Without extensive cross training, self-teaching, or other managerial attention, those folks are at risk when it comes to skill.

Second, managers aren’t rational. They’re not all knowing and all seeing. Multiple streams of research show they generally see things the way they saw them yesterday, and are therefore predictably blind to new opportunities, existential external threats, and internal friction in their own firms. Bob Sutton, one of my intellectual heroes, has a new book coming out about this last topic with Huggy Rao. It’s stunning how mucked up organizations can get while persisting for decades. Beyond this, managers and other professionals are satisfied with small, quick wins with new technology. In 1991 we got an absolutely superb (and woefully overlooked) piece of research on this from Marcie Tyre and Wanda Orlikowski. They found that after just 12% of the expected implementation time, most organizations simply stopped changing because they’d gotten a significant productivity boost. So they left most of what they could have gotten out of the tech on the table, and certainly didn’t look into collateral effects like deskilling.

The third point is that genAI is mostly getting rolled out by individuals, and managers may not notice. Over the last 18 months, we’ve tossed a free, internet-accessible version of the most powerful general purpose technology in a generation into the hands of billions of working adults. Some of us ignored it. Some of us wasted our time with it. But many millions of us have used it to do more, better than we could before, because producing content takes less skill and time than it used to. And we don’t need to tell anyone that we’re using genAI, nor is there a way to tell if we are: this is what Ethan Mollick calls the “secret cyborg” problem. The problem for reallocating labor to avoid deskilling in this case is that managers can’t optimize what they can’t detect, so it will take them quite some time to realize why some of their employees, contractors, colleagues, or even bosses are producing a lot more quality work, faster. They may never recognize it, and so leave valuable talent cranking away on increasingly rote tasks when they could be allocating it to more complicated work.

The stereotypical economist (I don’t know many of them, to be honest - the ones I work with are very well attuned to these dynamics) might argue that organizations like this are doomed, and in the long run they’ll be replaced by more efficient ones that allocate workers in skill-enhancing ways. But they know from one of their own brightest stars that this won’t much matter for you, your colleagues, your kids, or your friends:

‘The long run is a misleading guide to current affairs. In the long run we are all dead,’

- John Maynard Keynes, A Tract on Monetary Reform (1923!).

The upshot of all this is that unless we take proactive steps to avoid it, we’re all at serious risk of becoming like the push-button operators of that automated injection molding machine. More throughput, better quality, sure - but stuck in that situation in ways that subtly degrade our skill the longer we remain. And because we’re all driving our own boats this time - management-directed efforts to integrate the technology are quite rare and very small scale - we’re also on our own when it comes to protecting our journey to more valuable and satisfying skill.

SOS - Save Our Skills!

What can we do to avoid deskilling ourselves as we use genAI?

For today, I’m going to focus on the immediate, and the personal. Thankfully, there’s a simple step you can take right now that will drag you away from the slippery slope of deskilling. The more you do this, the better off you’ll be.

This involves using the “custom instructions” capability that OpenAI has rolled out to all users - free and paid. Basically you’re going to tell ChatGPT to nudge you towards more skill in every interaction.

Here’s how to do it:

Log in to chat.openai.com, and go down to the bottom left hand side of the screen to click on your settings. You’ll see the custom instructions option right there:

Click on that, and you’ll see a window like this:

This is a place where you can use natural language to program ChatGPT on how to program you. That’s right. You can tell it what your values, preferences, beliefs, or goals are in the top box, and give ChatGPT specific tactics in the second box to make your actions move your work closer to your values.

It just so happens that - if you buy the article above - you value your lifelong journey to skill, and you want to enhance that while supercharging your work with generative AI. So tell it that!

Starting about four months ago, here’s what’s I put in my custom instruction boxes - both are a work in progress, so have changed slightly as I’ve gone along:

My “about me” box:
“I'm looking to learn and build skills. I need a healthy amount of challenge (working near the edge of my skill), complexity (engaging with the broader context for my task to discover new skills I wasn't aware of), and connection (warm bonds of trust and respect).”

My “How would you like ChatGPT to respond” box:
“Help keep me challenged: ask me if I can confidently do what I'm asking you to do. If I say I can't, suggest a portion of the work I could do on my own instead of asking you to do it. If I say I can do it, inform me of more comprehensive or elegant solutions, and ask if I want to learn how to produce them.

Help me engage with complexity: give me a brief overview of skills, work, roles, and other contextual information related to my task that I might not get if I just got the job done. Also remind me at the end of our task to reflect on it later, to cement my learning.

Help me build human connection: right off the bat, suggest ways I might build bonds of trust and respect with others interested in what I'm doing. Either peers, people who know less than me, or people who know more than me. Be very creative here, and remind me of the importance of this at the end of our work.

Also, offer to debrief complex interactions just before they're concluded. If I agree, begin by offering what each of us did that was particularly helpful, then offer what you think each of us could have done differently to make things even more effective. Then ask me for my reactions to your assessment, as well as my own assessment.”

[I have more text in this box, by the way, I’ll leave that for another post].

Practically, this means I have to read a few more lines of text after my interactions with ChatGPT. Then I have to make a couple extra choices: do I want to learn more, or do I just want my results? Do I want to understand how the injection molding machine works, or how inventory control is done, or more about the different materials the machine works with, or about labor process… or do I just want to produce my widgets? It’s my choice, so while it can be mildly annoying at times, I feel like I’m in control of my skills journey.

And the debrief component has turned out to be very helpful. That should come as no surprise: we have a mountain of research that shows debriefing aids individual, pair, and team-based learning. Why not debrief with your computational prosthesis? Here I’ve learned how to optimize prompts, avoid hallucinations, and to skip steps altogether. Those are all “meta” skills - skills required to build more skills as I use the system.

I’ve learned new skills in coding, data science, cooking, experience design, musicology, and even natural language processing (a form of AI) this way, and I learn one or two things like this every couple of weeks. My learning curve is speeding up, not slowing down. And I take a moment about once every couple of weeks to tweak the contents of these boxes in search of better results. Have you tried anything like this? Drop your experiments in the comments!

Everyone for themselves is no way to win

I’ve given you a right-here, right-now tactic that you can use to protect your current skills and nudge yourself to more and better ones. But we need to do far better than this if humanity is going to survive the tsunami of productivity that’s breaking on our shores. We need to organize and invest here to realize the skills boon that we need to contend with all the change we’re creating.

Eventually I’ll offer some tactics and principles here, but I believe the atomic unit of progress sits just above. Managers, how could you design your organizations to nudge your people towards skills they desire, just by using genAI? Alternatively, what if you asked genAI application vendors to design their tech to do the same? And technologists, try treating this as the huge market opportunity that it is: how could you design your systems to nudge us all towards the skills we’d like to build, instead of simply being the operators of your ever-more sophisticated and capable models?

We need all hands on deck to avoid a massive wave of deskilling. For now, I suggest you get your hands on your custom instruction oars and start to pull.

I don’t need to tell you the world of work is changing. We all know it.

ChatGPT, Bing, Anthropic’s Claude, and Google’s Bard (and soon, Google’s Gemini) are free, and available to anyone with an internet connection. Image generation technologies like Midjourney and Dall-E are likewise turned on. And as I type this, OpenAI is rolling out its new multimodal ChatGPT (scroll down to the semi-viral “bike seat” video for maximum wow) - a system capable of receiving, processing, and producing an integrated blend of text, human-like speech, and images. And most of this capability is free, and relies on the chat-based interface that has become intuitive for billions of us, thanks to smartphones.

This is to say nothing of the long list of improving hardware out there - like processors (e.g., GPUs), sensors (e.g., cameras, microphones, accelerometers), actuators, motors, and batteries. In general, hardware is getting dramatically cheaper, better, and more reliable. This, coupled with steady improvements in planning and control software (increasingly related to the above) means we’re making robots - embodied computers that can sense, plan, and then act in the physical world - that can independently handle a far wider range of changing circumstances. For the whiz-bang end of this, take a look at my running list of practical humanoid robots. They’re not ready for primetime, but they’re getting *very* close:

All of this is being put to work… At work.

If you have internet access, you’re on the ride

We are dragging these technologies into the everyday work of millions of people. In some cases, this is in the form of organizations making strategic decisions: for example Price-Waterhouse Coopers (aka PwC), one of the world’s biggest consultancies, recently announced it would invest $1bn to “expand and scale AI capabilities.” But it’s also personal: hundreds of millions of us are going on the web and just trying these technologies out on whatever work problems suit our fancy. This is especially easy with web-connected generative AI because it’s impossible to tell whether someone’s used it if all you get is the output.

Aside from the advent of the internet, this is the only time we’ve lived through the advent of a general purpose technology. By definition, these are tools that are relevant to - and will eventually be insinuated into - most jobs around the world.

If even the conservative version of recent projections are correct, 2.6 billion of us could put generative AI to use in at least 10% of our jobs in ways that give us significant productivity boost. And a series of studies (here’s one recent one of consultants) show that those of us with low skill at at task stand to improve our productivity more than those who have high skill. Given these incentives and the eye-popping stats on usage (180m for ChatGPT alone) it’s a reasonable bet that millions of us are out there, putting genAI to work while organizations figure out how to do so systematically. And none of us quite knows what we’re doing - the clear picture of how to productively integrate general purpose technologies into organizations and systematic work processes only becomes available in hindsight. If the internet’s any lesson, the genAI integration will probably take a decade. Maybe more.

The wildness is about learning

Will jobs get lost as a result of this once-in-a-generation change? Sure. Will some get created? Sure. A big deal for those individuals affected.

I’ll write about that more later, but for now: job gains and losses are a tiny, tiny deal for our species compared to job change.

2.6 billion of us probably need to learn to do our jobs differently, starting more or less right now. Compare that to the hundreds of thousands of us who lose a job each year because of new forms of automation.

This is the Wild World of Work. Where we’re rebuilding the proverbial plane with new tools and techniques as we’re flying it. A generation of people lived through the arrival of electricity, then learned to do their jobs with electricity in the mix. After the shock (sorry, not sorry) and awe, new tools and techniques were developed, and we all had to learn them. Soon after, we figured out telephony. Then the transistor. In each case, a “raw” general purpose technology had to be adapted to be useful for a real problem, and that involved developing specific tools and related techniques, and then everyone - as in billions of us - had to learn to use them. Often the change wasn’t huge, and folks just seem to figure it out. After all, there weren’t really economy-scale retraining programs to accompany each of those general purpose technologies, and we did okay. Why not the same now?

…and we’re breaking the best school we’ve got

If history is any guide, organizations and institutions like schools and universities will figure out what to teach us about intelligent technologies after a decade - if we’re lucky.

So it’s up to us, and we’re relying on the taken-for-granted process of vicarious learning that’s unfolded between experts and novices for thousands of years.

In the rough-and-tumble of our everyday work, we’ll just try to figure these out. We’ll get a little help from someone who knows a bit more, often in exchange for helping them on something. We’ve known since at least the late 80s that this is how learning on the job happens - it’s informal and just-in-time, given a task to do. And post-books, TV and the internet, this doesn’t need to be with a coworker, with someone physically present, or even a live interaction: thousands of folks already share ChatGPT tips and tricks on this subreddit, for example.

The beauty of this process is that just by working, the novice builds skill, and both parties build a trusting bond. Of course it doesn’t always work, but it’s worked well enough for us to build and transfer skill since before recorded history. This is the best way to explain how we perfected multi-part stone tools like axes and spears. It’s certainly needed to explain the nested expertise to make the bow - our first stored energy weapon - about 160,000 years ago. That’s right about when we invented language. This process runs deep. It’s in the bones of every culture. It’s Normal. Good. Effective. Reliable. In a very real way it’s the bedrock of our civilization.

If you’re here because you already know my work, you know the hidden, troubling punchline that I shared on the Ted stage: we’re putting intelligent technologies to work in ways that hurt this vicarious, informal, on the job learning process. Intelligent technologies (ChatGPT is a great example) allow a single person to be much more “self serve” at work - they can get more done, faster, with less help. We take that deal. Productivity is hard to turn down.

But that means that the next person down the line - a novice trying to build skill and get ahead - is less necessary to get the job done, and doesn’t get as much (or any) chance to learn on the job. They also don’t get to build trust and respect with that expert colleague, which hurts both parties well beyond skill.

This is a massive problem for all of us. We’ll have to learn better, faster than we could before to adapting to work involving intelligent technologies - and we’re hamstringing our go-to method for that learning right when we need it most.

So, really, it’s a wild, wild world of work out there

It’s wild on the surface. Generative AI and advanced robotics inspire awe, fear, curiosity, anger - the whole gamut. And figuring out how to get tasks done with them is at once cool and tough.

But it’s wild down deep, too. We’re taking short-term deals with these technologies and selling our future down the river. On the skills front, we’re hollowing out our professions, occupations, organizations, and careers without quite realizing it.

Now is the time for the opposite. Now is the time to enhance human ability, just as we find good ways to put all this intelligent technologies to use.

We've known for decades that things usually go poorly when we first handle new technologies. Workers end up on the short end of the stick, organizations spend way more money and time trying to figure the technology out, and society misses out on huge opportunities and can pay other dramatic intangible costs.

But it's not always true - somewhere, someone is finding a much better way forward. They're just rare, and their solutions are incomplete. My research shows this is true with intelligent technologies, just as it has been since the bow.

I am focused on finding these positive needles in the negative haystack, and testing out which ones could be useful for us all. Beyond publishing papers, it’s now clear to me that I need to share these with the broader world, and to engage the diverse community of concerned researchers, practitioners, policymakers, and workers that have different-but-equal stakes in bending the arc of human ability in a healthier direction.

So here we go: let’s explore this wild world of work. I look forward to learning together.

Share

Subscribe now

Leave a comment