Crunch culture in the gaming industry is destroying developers and a detriment to the games themselves
In this episode, we talk about Google’s move to put their own chips in Pixels and Chromebooks, and notable items in GitHub’s “State of the Octoverse.” Then we speak with Nathan Grayson, senior reporter at Kotaku and co-host of Kotaku’s Splitscreen, about crunch culture in game development and the differences between a company like Supergiant Games and CD Projekt Red, the maker of the newly released and highly anticipated RPG, Cyberpunk 2077. Finally, we chat with Julien Cornebise, an honorary associate professor at University College London and a former researcher with DeepMind, Google’s A.I. lab, about Google’s firing of Timnit Gebru, a co-leader of Google’s Ethical A.I. team, who said she was fired after she sent an email criticizing the company’s efforts to hire more minorities as well as biases in their A.I.
Josh Puetz is Principal Software Engineer at Forem.
Vaidehi Joshi is a software engineer, creator of the Base.cs blog series, and co-host of the Base.cs podcast.
Nathan Grayson is a senior reporter at Kotaku who primarily focuses on streaming, labor, and PC gaming. He also cohosts the Kotaku Splitscreen podcast and is working on a book tentatively titled "STREAMERS" to be published by Atria/Simon & Schuster in the future.
Julien is a mathematician, scientist and coder, was the 5th research at DeepMind from 2012 working on their early algorithms, and co-created their Health Research team. He left in 2016 to start helping NGOs and charities with machine learning, and for that created in 2018 Element AI's "AI for Good" team and their London office. From ASMx86 to convergence theorems, from Google to Amnesty International and academia, he believes in depth and breadth, and that individual responsibility never dilutes in an organization: it multiplies. We must do better!
[00:00:10] JP: Welcome to DevNews, the news show for developers by developers, where we cover the latest in the world of tech. I’m Josh Puetz, Principal Engineer at Forem.
[00:00:18] VJ: And I’m Vaidehi Joshi, Leader Product Engineer at Forem, setting in for Saron Yitbarek.
[00:00:22] JP: This week, we’re talking about Google’s move to put their own chips in Pixels and Chromebooks, and notable items in GitHub’s state of the Octoverse Report.
[00:00:30] VJ: And then we’ll speak with Nathan Grayson, Senior Reporter at Kotaku and Cohost of Kotaku’s Splitscreen, about crunch culture in game development and the differences between a company like Supergiant Games, and CD Projekt Red, the maker of the newly released and highly anticipated RPG, Cyberpunk 2077.
[00:00:49] NG: There had been people on the team working like nights and weekends for like a year.
[00:00:54] JP: Then we’ll chat with Julien Cornebise, an Honorary Associate Professor at University College London and a Former Researcher with DeepMind, Google’s AI Lab, about Google’s firing of Timnit Gebru. a co-leader of Google’s Ethical AI Team, who said she was fired after she sent an email criticizing the company’s efforts to hire more minorities, as well as their biases in their AI.
[00:01:14] JC: That was really bizarre essentially. I mean, everyone’s position was, “What the flying F?”
[00:01:23] JP: So in the past, we’ve talked about Apple’s continued ARM race to put ARM chips in all of their Macs. Get it? Got it. I know that was corny. The first products which were announced last month that feature the M1 processor and feature dramatically better performance and battery life than their existing models with Intel processors. We also recently had Ish ShaBazz, an iOS Engineer and Creator of the Capsicum App on the show to talk about the significance of this announcement.
[00:01:49] IS: Initially, I was like, “Wow! Intel is really kind of lagging here.” If you track like the Geekbench scores, the iPhone SE, which is thought to be like the cheap iPhone or the budget iPhone, is like outperforming the 13-inch MacBook Pro from last year, a single core, which is kind of like embarrassing in a way like, “Wow. How did this happen?” According to Bloomberg, Apple is also reportedly working on a 32-core processor for their high-end Macs to outperform Intel’s fastest chips. And we could see those as early as late next year. Somewhat related, we’ve got news now that Google is moving to put their own chip in future pixel smartphones and Chromebooks. The chip is code-named Whitechapel, and it was designed in tandem with Samsung. They’re using a five-nanometer technology and it’ll include an eight-core ARM processor. According to the report, part of the chip will be dedicated to improving the performance and always on capabilities of Google’s assistant. We’ll have a link to the scoop by Axios in our show notes. We should note that Google’s current chips are made by Qualcomm, and much like Intel with Apple, this could spell really bad news for Qualcomm as a chip maker. Vaidehi, what do you think of this news? It seems like we’re seeing phone and computer makers trend towards making their own chips and trying to control more of the hardware as well as the software.
[00:03:06] VJ: It’s an interesting theme where you see companies sort of focusing on a product like the hardware and the software of it, but a lot of the times outsourcing some of those pieces and getting that from another company and now a lot of the companies that we’re talking about, like Apple and Google, their products are like so now well-known, and like the companies themselves have just become so much larger that I think they’ve now sort of pivoted towards doing everything in-house, which was really interesting because as you mentioned, yes, it gives you more control, but I think there’s definitely a financial incentive. Right? If you no longer are sourcing these things from external companies or contributors, now maybe you can get away with doing things in the hardware that you don’t necessarily even have to publicize because you’re doing it all in house. So I’m always really curious when people make that shift because it’s like, “Well, you were getting these chips from somewhere else for so long. What changed that now you find the financial incentive?” Or maybe there’s like an incentive from the product standpoint to start doing it in-house because that’s like not an easy change.
[00:04:15] JP: No.
[00:04:15] VJ: So for a company to make that change, I feel like there’s something lurking beneath the shadows. But that’s just my philosophy with all big tech companies. I always feel like something’s lurking in the shadows.
[00:04:26] JP: That’s a really good point that it’s not an easy or fast change. Apple started designing their own chips over 10 years ago, I believe. And they bought a chip manufacturer, P.A. Semi, and started designing their own chips quite a few years ago. And it just has been manifesting over the last couple of years with iPhone chips. And Google actually bought Motorola a number of years ago with an eye towards manufacturing their own chips. They didn’t ever actually manufacture their own chips. But I wonder if that Motorola acquisition has laid the groundwork for them to design chips that Samsung will manufacture. But in the 5 to 10-year range, both of these companies have been making moves to get into the space. It’s definitely not overnight. Yeah. That’d be definitely interesting to see how this shakes out, especially as Apple has set off such a bomb in the industry with their Apple Silicon chips and really I think we’ll see Intel on the defensive. I wonder if we’ll see the same with Qualcomm.
[00:05:22] VJ: Yeah. This week, GitHub published its State of the Octoverse Report. This year’s report is particularly interesting given the impact of the COVID-19 pandemic on developers and the report highlighted some of the changes that came because of the pandemic. The report found that overall there was a significant spike in open-source contributions, following the COVID shutdown in mid-March of this year and estimated that there was approximately a 20% increase in those open source contributions. And while developer work actually saw a downturn during the weekend hours, open source contributions jumped on the weekends. The report also observed a 25% increase in open source contributions and a 34% increase in the number of open source pull requests that were actually merged, which suggests that not only were more developers contributing to open source, but more of those contributions were actually being merged too.
[00:06:20] JP: Oh, interesting.
[00:07:14] JP: That’s crazy to see such a dramatic and measurable change in behavior as soon as the pandemic started and a lot of people were in lockdown.
[00:07:26] VJ: Yeah, absolutely. I think the thing that was so interesting to me is that the report mentioned that like a lot of contributions from developers that were using enterprise accounts, those are the ones that saw like a downturn on the weekends. So it kind of is interesting. It seems like developers were not working on work on the weekends as much during the pandemic, which you could draw some different conclusions from that, whatever that means. People just had like responsibilities or people wanted to unplug. But then at the same time, you have an uptake with open source contributions on the weekends. So maybe it’s like people found more time to contribute or people had nothing else to do. So they were like, “I’ll contribute to open source,” I guess.
[00:08:06] JP: Exactly. We’re not going to movies or concerts or traveling. I guess open source contributions are down on that list, but now they’re higher up. I find it really, really interesting. It’s great news for open source.
[00:08:20] VJ: Yeah. And I’m curious, will this trend continue when the pandemic is over? Do you think that the community got new open source contributors permanently? Or do you think that it’ll be like a temporary thing and when the pandemic is over, who knows? Whenever that is, people will just be like, “Okay, I don’t do that anymore. I’m going to go out to movies and whatever.”
[00:09:41] VJ: I really liked that philosophy. It’s kind of like a silver lining that like, yes, the pandemic happened, but maybe some people who would have been too intimidated to contribute before, maybe this was sort of like a forcing function to help them get into the open source community.
[00:09:59] JP: Yeah.
[00:09:59] VJ: That’s a very positive spin. I like it.
[00:10:01] JP: I mean, I think we’ll definitely see a decrease. When? I mean, let’s be clear. Once something resembling normal life resumes, I know myself I’m not going to be at home. Shunning friends and family and human contact at least for the first weekend.
[00:11:03] JP: It’s really interesting. I have to say like my beloved Ruby in the basement still. It’s okay, Ruby. I still love you. Shell scripting, C ++ and shell scripting higher than Ruby. Oh! Breaks my heart. Coming up next, we’re joined by Nathan Grayson, Senior Reporter at Kotaku and Cohost of Kotaku’s Splitscreen, to talk about crunch culture in game development, and at CD Projekt Red, makers of Cyberpunk 2077, which is releasing this week after this.
[00:11:54] JL: Triplebyte is a job search platform that allows you to take a coding quiz for a variety of tracks to identify your strengths, improve your skills, and help you find your dream job. The service is free for engineers and Triplebyte will even cover your flights and hotels for final interviews.
[00:12:09] SY: Vonage is a cloud communications platform that allows developers to integrate voice, video, and messaging into their applications using their communication APIs. Whether you’re wanting to build video calls into your app, create a Facebook bot or build applications on top of programmable phone numbers, you’ll have all the tools you need. Formally known as Nexmo, Vonage has you covered for all API communications projects. Sign up for an account at nexmo.dev/DEVNEWS2 and use promo code DEVNEWS2 for 10 euros of free credit. That’s D-E-V-N-E-W-S, in all caps, and the number 2, for 10 euros of free credit.
[00:12:54] JP: Here with us is Nathan Grayson, Senior Reporter at Kotaku and Cohost of Kotaku’s Splitscreen. Thanks for joining us.
[00:13:00] NG: No problem. Thanks for having me on.
[00:13:02] JP: So we’re talking today about crunch time, specifically how it relates to video game developers. And you’ve done some reporting on the topic. You wrote a piece in Kotaku entitled “The Secret of Success of Bastion, Pyre, And Hades: No Forced Crunch, Yes Forced Vacations”. Can you paint us a picture of what crunch culture is in game development?
[00:13:21] NG: So basically games are these giant, massive projects with a million different moving parts and a bunch of people kind of trying to all work in conjunction to realize them. And so what this results in, and a lot of cases, especially toward the end of development, it’s kind of a sustained period of what I think many would define as overwork, but basically just extended work time. So it could be an additional day, which was the case for Cyberpunk 2077 or nights and weekends. But in any case, the end result is that developers end up pushing themselves pretty hard in order to either hit some sort of milestone or get a game out the door. And the kind of pernicious element that I think results in the term “crunch culture” is that it’s not just a thing that happens during like mandated periods of time. The way the game development has kind of shaken out is that we now have a structure where a lot of people just sort of brunch indefinitely or there’s kind of an internal cultural pressure to crunch, even when it’s not necessarily mandated by the company. Just because, understandably, everyone wants to do their best work. They want to make the best game possible. But at the same time, you got to have limits, you got to take care of yourself. And there aren’t really that many safeguards in place at companies for that. There are cases in which somebody could easily say, “Okay, I’m going to work all weekend long on this project. I’m going to put in an obscene number of hours.” And there’s not really anyone or any instruction that says, “No, don’t do that.” Instead, there’s just a lot of internal pressures that say, “Yes, absolutely do that.” And do that at a number of points throughout development that are explicitly denoted and also just whenever.
[00:15:07] VJ: So a big offender of this crunch culture that you reported on is the game studio called CD Projekt Red, which just released its highly anticipated video game called Cyberpunk 2077, this week, which you just mentioned. Can you walk us through the story of this specific studio and its work culture?
[00:15:26] NG: So this is an interesting case in that previously CD Projekt had kind of made a promise to not crunch on Cyberpunk. They’d reiterated this promise a couple times over the course of the past, I think couple of years, which was admirable, right? Most game studios making games of this scale and Cyberpunk is an enormous game, absolutely gigantic, rivaling at least in terms of like the size of a production, like anything Hollywood has ever done. But yeah, so most games on that scale end up crunching at some point. Sometimes to hit milestones midway through development, sometimes to produce demos for tradeshows, and then often in the final sprint to get the game out the door. So it was cool to CD Projekt back then anyway, be like, “We’re not going to do that. We want to do this in a humane way, that respects the time and humanity of the people working for our company.” And then they sort of went back on that earlier this year and said, “Actually, we are going to have to crunch.” And not only that, when that news came out, there were also developers saying to Bloomberg, where my former coworker, Jason Schreier now works, that there had been people on the team working like nights and weekends for like a year. So it wasn’t just like now we’re going to start crunching. It was like also that there had been many people crunching for a while.
[00:16:45] VJ: Oh my gosh.
[00:16:46] NG: So this project that said it was going to be different turned out to not be different really at all in that regard. So that resulted in a lot of backlash and then things split off in a bunch of different directions from there, because for one, the game isn’t out yet, it’s coming out later this week, but there are copies in the hands of press. So people have been able to play it and kind of get a sense for what it’s like. And one of the main issues is that it is just overwhelmingly buggy.
[00:17:11] VJ: Oh, wow!
[00:17:11] NG: And so the idea was the crunch was going to result in fixing some of those bugs. That didn’t happen. And that sort of feeds into a broader criticism of crunch culture, which is that it isn’t even really effective for its stated goals that when you have a bunch of tired, overworked developers, instead of fewer bugs, you get more bugs because they miss things. Naturally, they’re human beings. So there’s that. And then also there’s this other perhaps equally pernicious element of this being this like massively hyped video game that a lot of people decided they are going to love before they ever got a chance to play it. And so you have this like brand loyalty identity thing happening among a lot of the online fan base. And so that led to people, like when the news of crunch first came out, I think there was kind of a united front of, “This is bad.” But over time, people are like, “Wait, this is a criticism, not just of this game and company, but like of what I perceive as my identity. So I have to defend this game and company and therefore also defend crunch.” And so you have like all of these interlocking ideas tied up in each other, and it’s just led to this very unfortunate and toxic culture around defending this company up to and including their bad work practices.
[00:18:26] JP: So CD Projekt Red has a number of issues. We should mention that they’ve had a lot of issues with transphobia in their games and their statements. Lots of bugs, as you mentioned and delays. So Cyberpunk 2077 was supposed to come out, I think it was three years ago. And then it was supposed to be last month. There’s even a meme going around on Twitter where you’ll see someone posts an important update and it’s just the yellow CD Projekt Red Cyberpunk 2077 statement. And you click and it’s like a rick roll. You mentioned that this crunch time actually leads to more bugs, more delays. Why do companies do it then?
[00:19:10] NG: I mean, it’s been a thing, like it’s one of those institutional practices that’s been around forever. It’s interesting because I was rereading my piece about Supergiant and Hades and everything earlier today, and one of the things that they specifically mentioned is that before forming Supergiant, Greg Kasavin and Amir Rao, who are two of the principal people at that company, worked at the EA Studio that was where an infamous incident called the EA Spouse Incident occurred. And so basically that was one of the first major incidences of whistleblowing in the video game industry when it came to work practices. The short version is that the wife of a developer who worked at Electronic Arts wrote this giant missive about how messed up their work practices were there and like how this person never saw their husband and it was awful. And that happened, like, I want to say in the earlier mid-2000s.
[00:20:06] VJ: Wow!
[00:20:06] NG: So it’s just something that people have been doing for a really long time. And I think that even though you end up with more bugs and things like that, it’s sort of one of these things where the way that AAA game development works and the scale of it and what they have to pull off means that a lot of companies backed themselves into a corner. And because they are beholden to so many financial interests, especially around release dates and things like that, they’re just like, “Well, we got to get this out the door. So either we go into crunch mode or we end up facing some sort of terrible financial penalty,” that probably mostly impacts the company itself and executives up top, but the people who end up paying that price are the developers.
[00:20:50] VJ: So the other thing I’m wondering is whether there have been any companies that are actually taking steps to rectify this crunch culture because we’ve sort of talked about how detrimental it is to employees and to the games themselves, but is anybody actually doing it right? Do you see any shifts in the industry to take things in the right direction, maybe even perhaps in response to the pandemic?
[00:21:15] NG: There are a couple of really good examples here and I think that they work well because, one, is kind of a smaller scale example, and the other is a larger scale example with some bumps in the road. So the smaller scale one is, as I mentioned earlier, Supergiant, the company who made Hades. Hades is a kind of Greek mythology inspired action role-playing roguelite game. A lot of descriptors basically run through hell repeatedly and try to kill your dad. It’s great fun. But yeah, so that game has been incredibly well received, kind of consensus game of the year for a lot of people at this point. And it was made in conditions that sound overall like they were pretty good and that they were more importantly, very, very mindful of the sorts of behaviors that can eventually lead to crunch. I interviewed them a little while back about this and they talked about how they started this company when they were kids or at least when they were like in their early 20s and they were very able at that time to like overwork and push themselves and do all this stuff. But as they’ve gotten older, people who work there have gotten spouses and families and other responsibilities and also they’re in their 30s and 40s, they can’t work that way anymore. And so they’ve developed a lot of like very, again, aware practices to avoid burning themselves out because this sort of game that I think Hades is, with the amount of detail that’s in it and the amount of clear like care and craft, you don’t get that from people who are overworked and tired and burnt out and uninspired. You need sustainability to create truly amazing games. So their sort of broader philosophy is that I think anything that leads to overwork, whether it’s during a period of “crunch” or not is just not something that you should drag the whole company into. So the way that they approach it is that they say like, “Okay, well, if one person at the company gets really excited about an idea, whether that’s for somebody who’s working on the soundtrack, a particular song or a level or area they’re designing, our mechanic or whatever, that person can go nuts if they want to. They can work as much as they want to on it. They are not allowed to drag everyone else into it.” So if it’s a Friday afternoon, no emails after 5:00 PM, period, even if it’s just like a little quick thing to say, like, “Oh, man, I just did this one thing and I think it’s really cool.” Because when you do that, there’s an implicit pressure for other people to join you and throw out their weekend to work on the game. So then on top of that, they also have, and I really enjoy this and I wish everyone would do this because I would definitely make use of it, they have mandatory 20 days off per year. So they have unlimited vacation time. But what they found is that people aren’t using that because it’s always like in your back pocket, right? To bring it back to video games, I don’t know if any of you do this, but there’s that thing in video games where like you have a bunch of items that you collect and then in battles, stuff you just never use them.
[00:24:11] JP: Never!
[00:24:12] NG: Yeah. It’s for later.
[00:24:13] JP: Later.
[00:24:13] NG: Yeah, it’s for when I really need it. And then you just never do it. And so that’s what they were finding with vacation at Supergiant. They were like, “Oh, you can take as many days off as you want to,” but no one is. So they basically said, “Okay, now you have to. Now you have 20 days out of the year where you absolutely must not be working.” So the idea there is maybe you use it every Friday for a while or maybe you take almost a month off at the end of the year. So just things like that that I think target these little areas that eventually cascade into like proper crunch and overwork. But yeah, so there’s Supergiant. And then the other really good example or at least interesting example is Respawn. Respawn Entertainment makes a game called Apex Legends. It’s a big Battle Royale type game in the mold of Fortnite, but more realistic graphics, futuristic setting, comes from a series called Titanfall that has giant robots. They made a commitment earlier this year to avoiding crunch, even when it comes to like running a live game.
[00:25:14] JP: That’s really interesting in terms of a live game situation.
[00:25:16] NG: Yeah. It’s very interesting because it also means that crunch is having to metamorphose to fit the new way that games are made, which is that a game comes out, but it’s not done because games are never done anymore. They come out and developers keep making content for them because they have these ongoing sustained communities and you want those people to keep playing the game and hopefully spending money on the game. So Respawn looked at the landscape around these sorts of games and especially Fortnite. Fortnite puts out new stuff like every week. They never stop it as bonkers. And wouldn’t it, they crunched to do that. Respawn looked at that and said, “We don’t want to be the company that crunches to sustain our game. So even if it means that our updates are going to take a little bit longer than other games in the genre, we’re going to treat people more humanely.” So they said that, and that was of course, very well received because you rarely hear that from big studios. But they didn’t really manage to entirely pull it off this year. And part of that was apparently because of the pandemic. So I think a few months ago, there was a post on like Glassdoor where a Respawn employee basically said like, “Yeah, past couple months we’ve been crunching like crazy.” It kind of blows. And so after that initial promise to avoid crunch, of course, people went back to the higher up at Respawn who said that and went like, “Hey, what’s the deal?” And that person was like, “Yeah. So we wanted to avoid crunch and we’ve resumed trying to avoid it, but there was a rocky bit in the middle when the pandemic first started and we had to transition to everybody working from home. There’s just a lot that we didn’t know how to handle that was unprecedented. So we kind of had to really go for it for a little bit there to bring everything back up to speed.” So as far as I know, they’re back to avoiding crunch, but it’s also one of those things where when you hear somebody who runs an entire studio, say that there’s no crunch, you should be very skeptical because of course they’d say that. They benefit from that. You want to hear from the rank and file people who may or may not feel an implicit pressure to crunch even as their boss says, “No, we don’t crunch. We never crunch.”
[00:27:26] JP: Well, I seriously could talk about this all day, but Levi will kill me. But to wrap up, is there anything you’d like to cover that we didn’t get time to talk about?
[00:27:38] NG: I think that what I find most interesting now is that there was, for a few years, a very kind of concerted backlash to the idea of crunch, I think, within the games community, and that was good to see. And now, regrettably, there’s kind of a backlash to the backlash because video games are ultimately a consumer-driven industry, one where a lot of people look at developers as a catalyst for the creation of things that they like. And so they often do not see the human cost that goes into their video games. And now because of things like Cyberpunk, they have managed to find justifications to not care, to be like, “Yes, I acknowledge that they’re crunching, but ultimately what matters more to me is that I get my video game. Whatever mental gymnastics I have to do to justify that, even though it is a fairly cruel point of view, I will do.” That’s been disheartening to see, to say the least, but I think that as long as there’s money to be made, a lot of marketing and studios will overlook the toll that crunch takes and a lot of gamers will be happy to go along with it because it means they get their video game.
[00:28:46] JP: Well, Nathan, thank you so much for joining us today.
[00:28:48] NG: Thank you for having me.
[00:28:55] JP: Coming up next, we talk with Julien Cornebise, Honorary Associate Professor at University College London, Former Director of Researcher at Element AI, and a former researcher with Google’s DeepMind, about Google’s firing of Timnit Gebru. a co-leader of Google’s Ethical AI Team, and the impact it may have on developers and the wider tech industry after this.
[00:29:28] JL: Join over 200,000 top engineers who have used Triplebyte to find their dream job. Triplebyte shows your potential based on proven technical skills by having you take a coding quiz from a variety of tracks and helping you identify high growth opportunities and getting your foot in the door with their recommendation. It’s also free for engineers, since companies pay Triplebyte to make their hiring process more efficient.
[00:29:50] SY: Vonage is a cloud communications platform that allows developers to integrate voice, video, and messaging into their applications using their communication APIs. Whether you’re wanting to build video calls into your app, create a Facebook bot or build applications on top of programmable phone numbers, you’ll have all the tools you need. Formally known as Nexmo, Vonage has you covered for all API communications projects. Sign up for an account at nexmo.dev/DEVNEWS2 and use promo code DEVNEWS2 for 10 euros of free credit. That’s D-E-V-N-E-W-S, in all caps, and the number 2, for 10 euros of free credit.
[00:30:37] JP: Now joining us is Julien Cornebise, Honorary Associate Professor at University College London, Former Director of Research at Element AI, and a former researcher with Google’s DeepMind. Thank you for being here.
[00:30:48] JC: Thank you very much for having me. And sorry, you’re stuck with me. There’s many people who could speak about this topic who suddenly were not necessarily available or are completely swamped who do not dare speaking. So I’m afraid you’re stuck again with a white male in his 30s. Apologies for that.
[00:31:05] VJ: So I want to dig into the story of the week, really, which is that this week, Timnit Gebru, a co-leader of Google’s Ethical AI team, said that they were fired by Google after an email that they sent criticizing the company’s efforts to hire more minorities as well as their biases in their AI. Can you give us a little bit more context around this news?
[00:31:29] JC: This came as a shocking news and in a lot of ways, because Timnit is extremely respected in the field and is really a shining star. And because the person who was involved in the firing is none other than Jeff Dean, who has also been an inspiration to many developers and researchers. So there’s this clash here that just shocks. So what has been widely reported, Timnit wrote a research paper with other co-authors who wrote through the internal review said, “Well, it’s a little short. You’re putting it for internal review just a day before submission,” which is what everyone is doing. And eventually the internal admission was rescinded saying, “Well, no. Actually, no, we said you could, but you can’t publish it, but we can’t tell you why.” And it’s actually HR and legal who reached out to Timnit, which is very uncommon, and asking her to retract the paper. Timnit went to answer, “Well, look, okay. No, that’s just not going to cut it. If you ask me to retract, I need to know what are the critiques and where are the critiques from,” because that’s how peer review of an internal normally works. And if we don’t do this and that, these two conditions, then we can work. The email is public. I’m happy to work towards an end date, to which the answer was, “We can’t accept these two conditions. And rather than working to an end date, you are suspended immediately. Thanks for resigning. We accept your resignation,” which came as a shock really to everyone involved, I believe, because that did not even go through Samy Bengio, direct manager, who is himself a very appreciated senior researcher in AI, in machine learning. So that was really bizarre essentially. I mean, everyone’s position was, “What the flying F? What happened there? What happened?” Shaking our heads in disbelief.
[00:33:24] JP: So in the past, you were a researcher working at Google. You mentioned this was shocking in the AI industry as a whole. Is this shocking in the context of Google? And specifically, I wanted to ask, there’s been a lot of discussion about this two-week review period. Was that a normal course of review at Google in your opinion? Or does it seem like this is an unusual course of action based on your experience working at Google?
[00:33:50] JC: So looking at this point in particular, the two weeks’ review period and the tool that we call “PubApprove” internally, this is absolutely standard to submit your paper through PubApprove. It says, “Please submit in weeks.” And it’s absolutely standard that everyone submits it two days before deadline. If anything here, Timnit and her co-authors have gone beyond the line of duty and that they gave heads up to the PR, the public relations team at Google about, “Hey, we’re going to be working on this topic.” They gave notice of that month in advance, which is no one requires you to. The submission through PubApprove, everyone does at the last minute because you submit your stuff for conferences at the very last minute, you work until the end of the deadline, everyone’s guilty of that. It’s terrible. It’s like people at crunch mode in video games. We shouldn’t be doing that. We all do it. And we all do it to ourselves. I mean, it’s madness. But that’s what we do. So it’s commonly accepted that, yeah, you’re going to submit it just two days before, one day before. That is common. What it is done for is it is not to check so much the quality, but making sure that no sensitive information about the internal workings of Google are published. For example, if you start to say, “Here’s how many servers we have in all our data centers,” yeah, that does not need to be published. That actually has to not be published. The only check of quality there is, is just making sure you do not have another excited young researcher wet behind the ears, once you send a paper and he’s going to make a fool of himself because the paper is not of a decent quality. Absolutely not the case for Timnit. Timnit is a researcher that is widely established, whose work has been path leading. She has reached into her multiple co-authors, some at Google, some outside of Google. The article has a more than 120 citations. So heavily referenced. I’ve read the paper. It’s a great research paper. And ironically, it addresses problems that are really accepted in the community about the weaknesses of these large language models. There’s nothing shocking in it. There’s nothing that would warrant being fired or being pulled from a review process. I think the extra thing that made this really abnormal that normally in the review process, yes, you’ll have colleagues reviewing your paper for these sensitivity aspect and then you’ll get the feedback, straight, conversation from Googler to Googler, simple. Here, that’s not what happened. What happened was, as far as I could understand it, that HR and legal reached out to Timnit about our paper and saying, “Oh, hold on. Yes. We know someone on PubApprove gave you permission, but actually now that we’ve looked at it, you really cannot publish that and you need to remove it.” So that’s HR/legal stepping in. When has that been the case for paper? Nowhere. I mean, I’m not the only one saying it. You can see it on Twitter that everyone is ablaze with Google saying exactly that.
[00:36:57] VJ: So that’s really interesting. So as far as you know, has anything along these lines ever happened in the industry, especially in the field of AI ever before?
[00:37:09] JC: Not that I’m aware of. I’ve never heard of it happening. I know some places published less than others. If you go to Apple, you’ll be publishing much less than if you go to Google. But that’s down to just habits and culture internally. Google as being the number one institute in terms of number of publications in all the major machine learning and AI conferences. So these to me has never happened before, to my knowledge.
[00:37:38] JP: What do you think the impact will be on developers in tech, in general, after someone like Timnit Gebru being fired after giving a critical opinion of her employer?
[00:37:50] JC: So first, I do not think that our article was a critical opinion of our employer. Google is not named. No business is being singled out. It is purely talking about weaknesses in algorithms. And the example that her and our co-authors put forward are actually examples from many different research institutes. They talk about GPT-3 for open AI who just got the best Paper Award at NeurIPS, by the way, and that’s not a Google thing. The paper is not at all a critique of Google. It’s so happened that Google, using cutting-edge algorithms, does use these algorithms too. But so you know, so do Facebooks, so do Microsoft, so do anyone. And again, no single company is being singled out in this article. Google originally said that she was being let go because of her paper. Apparently, reports of an internal meeting that happened yesterday to clarify the situation with the employees at Google, the management was explaining, “Well, it’s actually not fully about the paper. It’s about the email she sent after trying to request and demand that we give her explanations of why we wanted to her to retract, et cetera, et cetera.” There’s been a shifting set of explanations of why she was taken out.
[00:39:07] JP: I was going to ask, if her paper is talking about algorithmic use in the industry in general, why is Google coming down so hard on her for this paper?
[00:39:19] JC: So that’s where there’s a connection with every developer out there, and that is my personal take there. Timnit need is a tall poppy. She is someone who really speaks her mind, who has been fantastic. That is a requisite for the kind of work she’s doing, was not afraid to call a spade a spade, and she’s brilliant. Are brilliant people the kind of complacent and quiet employees? No, they’re not. Can a company handle that? You’d assume that Google, from its reputation and that the claim that he made, should be able to handle absolutely brilliant people who speak freely. And to me, that’s really the signal here. More chilling than just what happened to Timnit. And full disclaimer, she’s a friend. I’m really horrified to see what’s happening to her personally. But looking beyond the persons, it is what signal is this sending to any dev, any tech employee who wants to speak up at any point? Or say, “Hey, the internal process here is flawed. I deserve to know what are the critiques. I deserve to discuss.” It’s essentially saying don’t raise your head. There’s two aspects. There’s the aspect towards the developers and towards the ethics research. But if we focus here on the developers and the researchers, this is, okay, let’s make an example of this one. And this is absolutely scary because you have to put that in parallel with much more I would say senior because Timnit is incredibly established and you have to factor that she’s also super young. She passed her PhD a few years ago. She has the renown of a much more senior researcher. But now you compare that with say the treatment of, let’s pick one example from Google. Andy Rubin, the inventor of Android. We all know what happened, terrible, terrible behavior that are acceptable nowhere, yet being thanked away with millions of package. There are really two different ways it’s being put depending on where you are on the employment scale. And for developers, there are two futures here. One is we all get completely scared and we all shut up and cower and say, “Well, okay, I’m just here to do code and the rest is beyond my remit.” This is the nightmare scenario, because if you’ll do that and we all just shut up, we get into a position where, well, we basically condone terrible behaviors. I’m more optimistic. I want to believe, and I know I sound like Fox Mulder on this one, but I want to believe that in front of us, there’s a path where we all realize, “Holy cow! That is not acceptable,” how we are treated as employees, how large organizations lose their sense of humanity and make other people, including good people in these organizations lose their own sense of humanity and we start to react.
[00:42:18] VJ: That makes a lot of sense. And I’m kind of wondering now, is it possible to be employed by a company and do research on that company’s products and also be free to be totally impartial? To me, it seems like there’s maybe many different conflicts of interests going on there. Do you think that that’s a feasible goal?
[00:42:39] JC: It’s an excellent question. And let me slightly rephrase here, because again, in this example, Timnit was not doing research on her company’s products. She was doing research on algorithms that are used widely. So the problem is even worse in a way. Let me make a comparison after which I’m pretty sure I’m not going to be able to ever get a job at Google. But you know what? Some things are meant to be said. if you look at the history of selective publishing, so you run different studies and you decide which ones you are comfortable making public, there are terrible precedence for that. There is a tobacco industry in the ’50s doing studies and only publishing those that say, “Oh, no, there’s no cancer here. Nothing to see.” We discovered later that, well, there’s all these pilot studies that has not been published and that showed there was cancer rates. There is the oil and natural gas and fossil fuel industry in the ’70s who we now discover had internal reports saying, “Well, we might be putting a whole lot of CO2 out there and there’s a risk of warming of the planet.” And that is now documented. And these were not published. There was a reason why in medical research, the gold standard is now to publish your study protocol before you lead the study, because you register, you say, “I’m going to study this.” And if nothing is published, that’s really weird. People can find it. That’s a way to ensure that there is an even reporting of the results of any medical study. This move by Google really worries me in terms of these historic echoes that it is bringing and it’s bringing it about AI. And we may not be paying the costs tomorrow, but we are seeing with all these previews, these story parallels, we are seeing the cost, gigantic costs over decades. And that’s what is worrying me right now in terms of what can happen when we have this so much ethical research as much as machine learning research being done in the industry.
[00:44:48] JP: Is there anything else we didn’t cover today that you’d like to talk about?
[00:44:52] JC: I think for all developers, engineers, researchers out there, I believe we have a duty to think what are going to be the uses of what we develop. It is true that we could say, “Well, what I’m doing, if I don’t do it, someone else will.” And that is true, yet it will not be your hand in it. And by not working on it, you will be slowing it down in a way. We have this responsibility to think and to even if you don’t know, that’s fine. I mean, just look around, try to analyze the structure in which you are doing your work, analyze what are the different incentives of the structure, what are the power struggles that are in there. Essentially, just like as an engineer, you analyze a software, you analyze an algorithm, you analyze the library, analyze the ecosystem in which you were playing. Keep in mind, these are humans. So we are less rational with our lines of code, but you can start to build the model for that. And when you start to look at how the things are working around you, then you can say, “Okay, I can act with my responsibility.” We are at the frontline of the social impact that our tools can have, that technology can have. We are developing it. We are the best place to say, “Ooh, hold on. Something is not quite right here.” We’re not the only ones, but we have a really strong role to play there.
[00:46:28] JP: Well, thank you so much for joining us today.
[00:46:30] JC: Thank you very much for having me.
[00:46:44] JP: Thank you for listening to DevNews. This show is produced and mixed by Levi Sharpe. Special thanks goes out to Vaidehi Joshi for joining us as a guest host this week. Editorial oversight by Peter Frank, Ben Halpern, and Jess Lee. Our theme music is by Dan Powell. If you have any questions or comments, dial into our Google Voice at +1 (929) 500-1513. Or email us at firstname.lastname@example.org. Please rate and subscribe to this show on Apple Podcasts.