Can ChatGPT teach you Scrum?
Jason Malmstadt
Alright, so today, I thought we could talk about ChatGPT. So if you haven’t heard of ChatGPT, it’s this artificial intelligence/AI chat bot. I feel like you can’t look at anything on the internet or hear anything on the TV or radio without hearing somebody talking about it. Either it’s gonna take over all our jobs, or it’s ruining term papers or whatever. So, I thought, you know, being some Scrum trainers, we pride ourselves on knowing “the Scrum” pretty well. Why don’t we throw some Scrum questions at ChatGPT and see how it does? Maybe it’ll get some things right. Maybe not. What do you think?
Robb Pieper
I’m into it. I would have loved to have this tool when I was writing term papers and had nobody know about it because this thing is just…it’s insane. So yes, ask it some Scrum questions.
Jason Malmstadt
What do you think? What should we start with?
Robb Pieper
What is Scrum?
Jason Malmstadt
(Singing and typing to ChatGPT) “What is Scrum?” Baby don’t hurt me.
(ChatGPT Responds) "Scrum is a framework for managing projects, particularly software development projects developed to help teams work together more effectively and deliver high-quality products on time within budget."
Robb Pieper
Boooo!
Jason Malmstadt
I don’t like that so much. That’s pretty project-ey speak there. Okay.
“Scrum is an agile methodology.” (Buzzer Sound)
"Which means it's designed to be flexible and adaptable to changing requirements." Ok, that part’s ok.
"Agile development's an iterative process, meaning it involves repeating cycles of development testing and feedback." Ok.
"Scrum framework consists of several roles Scrum Master, Product Owner, and Development Team…" That’s older language. The new Scrum Guide doesn’t say ‘roles’ and also doesn’t say ‘Development Team,’ it says ‘Developers’.
"Scrum Master is responsible for facilitating the process…" Hmmm…I have some problems with that. "…ensuring the team is following the Scrum rules." So it’s saying the Scrum Master is essentially a Scrum Cop who’s going to hit you over the head with a Scrum Stick if you step outside the rules. I don’t like that so much.
"Product Owner is the person who represents the stakeholders responsible for vision and goals of the project." Okay, I’m on board with that conceptually.
Robb Pieper
They keep talking about ‘project’ and use it interchangeably with ‘product’.
Jason Malmstadt
Yeah, that’s not so great. "Development team…" (Again, outdated language) "…is a group of people who are actually doing the work of building the product." Ok.
"Scrum includes ceremonies."
All
Boooooo!
Jason Malmstadt
I’m gonna put up my hood and start chanting.
"Scrum includes ceremonies such as the Sprint Planning, Daily Scrum, Sprint Review, Sprint Retrospective, which are used to keep the team on track, and ensure everyone is aligned and work together effectively. Scrum was widely used in software development and has been successfully applied to other industries as well, including manufacturing, healthcare, and finance. If you're looking to know more detail about how Scrum works and its practices, it'd be best to look for the Certified Scrum Master Training and Certification Program."
Robb Pieper
What? Is ChatGPT now working for Scrum Alliance?
Jason Malmstadt
Apparently. Because we would as Professional Scrum Trainers with Scrum.org, we would probably recommend the Professional Scrum Master certification over the CSM (which is a can of tuna for another day). Okay, so what do you guys think? We’ve had some ‘Boos’, we’ve had some jeers. Overall thoughts on this first question?
Robb Pieper
I’ll go first, I don’t know if Greg’s got any ideas here, but this Agile methodology nonsense… It’s not an gile methodology. It’s a framework. It got quite a bit wrong, where I’m like, I wouldn’t give this a passing grade. So if someone were writing a Scrum term paper, this is getting thrown off my desk.
Jason Malmstadt
Ok. Greg, what do you think?
Gregory Crown
Well, I think it’s a pretty common response when you ask people what Scrum is, if they think they know what Scrum is, this is what you’re going to hear. I think you might get this response from typical project management-type environments. And I’m not going to say project managers per se, but typical project management environments would give a response similar to this. Again, there’s just missing some of the specifics, which actually do matter. And semantics do matter. So, yeah, we’re just we’re missing some of the core aspects here. But I think it’s fairly common. I don’t hear anything here that I haven’t heard before or a dozen times.
Jason Malmstadt
Yeah, I think those are my overall thoughts as well. I mean, my first glance is to say it’s getting core concepts, mostly right and it’s just getting some word choices wrong. Do those word choices really matter? But I agree with you, Greg, I think those word choices do really matter.
Like “Development Team” versus “Developers”. I’m not going to jump up and down over somebody about that. That’s a change the Scrum Guide made about two years ago to say “Developers” instead of “Development Team”. Fine, fair enough. But some of this project management mentality that you see seep into this, like the very first paragraph. It’s focusing on projects on time and within budget. That’s not how we measure the success of a Scrum team. We measure the success (by asking) are we delivering value? How frequently are we delivering value? Are we satisfying our customers? Are we learning and adapting?
Robb Pieper
And accomplishing goals?
Jason Malmstadt
Yeah. Are we meeting the goals? Product Goal, Sprint Goal, and things like that.
Robb Pieper
And are we adapting when we realize we’re not?
Jason Malmstadt
Yeah, because you can blindly move forward and meet something on time and on budget, and nobody likes it. Nobody wants it. Nobody buys it. Is that success? I wouldn’t say so.
Gregory Crown
I find it kind of shocking that there was no mention of empirical process control at all. Where’s empiricism? I think that’s a huge miss.
Robb Pieper
It’s also missing all of Scrum theory. Nothing in here about Lean Thinking, nothing in here about empirical process control, like you said.
Jason Malmstadt
Yeah, did it even mention anything about reducing risk? I don’t see that anywhere, either.
Robb Pieper
Nope.
Jason Malmstadt
No. Alright, so maybe 50/50 at best?
Robb Pieper
I’d give it a 60%. Like a D or an F.
Jason Malmstadt
Yeah. Alright. How about…What else should we ask it about Scrum? Since we were just talking about its lack of empirical process control… and actually without referring to it by name, that “on time on budget”, it’s really more defined process control. The two points that Iron Triangle. Let’s say “What makes a Scrum team successful?”
(ChatCPT responds) "Several factors contribute to the success of a Scrum team. Some key elements…" (Oooh, and it's giving us a list) "…clear goals and priorities. The team should have a clear understanding of the objectives as well as the priorities for each Sprint. This helps ensure that teams focus on the most important tasks, that everyone's working towards the same goals." I’m buying that so far.
"Strong collaboration and communication. Scrum teams rely heavily on effective collaboration and communication to be successful. Team members should be able to openly share information and ideas, provide constructive feedback to one another." Fair.
"Self-organization: Scrum teams should be self-organizing…" The current Scrum Guide would say ‘self-managing’, but let’s keep going. "…which means that the team members should be able to take ownership of their work make decisions without relying on a manager or leader to tell them what to do." Wow, it’s given us a long list here.
"Four: flexibility and adaptability…should be able to adapt to changing circumstances and requirements…should be able to handle uncertainty and change in a constructive and efficient way. Five: Empowered Product Owner. Product Owner as a representative of stakeholders and business, should be able to make key decisions on Product Backlog, prioritize user stories, effectively communicate, keep everybody on the same page." Okay.
"Six: Certified Scrum Master" You were doing so well, ChatGPT!
Robb Pieper
Booo.
Jason Malmstadt"Six: Certified Scrum Master to lead and guide the team, implementing Scrum framework and practices, removing any obstacles and keeping the team aligned to the Scrum values and principles." Ok, let’s come back to that one.
And, "Seven: Continuous Improvement. Scrum teams should be committed to continuous improvement, which me"ans that they should be constantly looking for ways to improve their processes, and practices to deliver better results."
"All in all, a successful Scrum team is one that's able to work together effectively, deliver high quality products on time and within budget." There's that project-ey the language again. "It is a team that's committed to continuous improvement, open communication, flexibility, clear goals, priorities and strong leadership."
Alright, what do you guys think?
Robb Pieper
I give this a passing grade. This is pretty strong, aside from it doesn’t really understand that a ‘Certified Scrum Master’ is actually a brand. So had they been more generic and just removed the word ‘certified’ that would have been correct. And then this whole “delivering quality products on time and within budget”, that’s the only other part I have an issue. But, that’s a couple of points off. Yeah, I mean, “on time, on budget” isn’t always the goal. I mean, sometimes you realize, hey, we’re going to be on time, but it’s the wrong thing. And I know, this is what you asked for, but things have changed, are you sure you still want this thing? And that’s really kind of the point of Scrum: adapting to those changes, not necessarily getting to where you thought you were gonna get on day one, but getting to the place you actually need to be at the end of this thing.
Jason Malmstadt
Right, which, it does highlight in Item Four: Flexibility and Adaptability. So it does do that, but then it kind of circles back to that sort of predetermined time and budget idea at the end in that summary. Alright, Greg, what do you think?
Gregory Crown
I’ll throw something out here: So I like the continuous improvement mention, I think that is a pretty important aspect. I know it was already kind of highlighted there a little bit, but Learning Organizations is a big topic now. If you aren’t one, that’s a problem. It’s been embedded in Scrum, and I like that it was highlighted. Overall, I would also say this gets a passing grade. There may be some specifics that it should try to avoid. But other than that, I’m pretty good on this.
Jason Malmstadt
Okay. Yeah, I would agree. I think overall, a B, I will give it as a grade? I mean, it’s given me a list of seven things. Clear goals and priorities: check. Strong collaboration and communication: check. Self-organization: the new Scrum Guide would say self-management, but check.
Robb Pieper
Well, to this tool’s defense, isn’t it based on data on the internet from prior to two years ago?
Jason Malmstadt
Yeah, I want to say like, 2021? Let’s ask it. (Typing to ChatGPT) “What’s the latest information you have in terms of time? How recent are your answers?”
Robb Pieper
Wonder if it knows…?
Jason Malmstadt
(ChatGPT responds) "My knowledge, cutoff date is 2021 and my last update was on that date." Okay.
Now it’s telling me what it is and how it’s a language model and it can’t access the internet right now. So even 2021, I mean, the latest Scrum Guide came out in November 2020.
Robb Pieper
Alright, well, then it should have these answers.
Jason Malmstadt
It really should. Yeah. But there’s a lot of language out there on the internet. I mean, Rob, I know, I’ve been in your classes before. And I’ve heard you say (and I’ve actually started co-opting because every great Scrum trainer steals from another trainer), I started saying in my classes, if you want to fail any of your certification exams, Google the questions and use those answers to what you find because there’s a lot of bad info on the internet and stuff that’s not really related to Professional Scrum. So some of these things that it’s finding, or that it’s using as its answer is probably from either older versions of the Scrum Guide when that was still the most recent, or I don’t know, just stuff that doesn’t align with the Scrum Guide, which is pretty popular as well.
Robb Pieper
Yeah, and I think that kind of drove some of the reasons why we’re kind of going through this exercise is: Can ChatGPT replace your Scrum trainer? And so far, I’m not thinking so. And if you don’t know enough to know the difference, you end up learning the wrong thing. Or you’ll learn outdated information or what happens to be repeated the most on the internet, things like that. And not always is it right.
Jason Malmstadt
Yeah. Any other…? I feel like we could get one more Scrum question in here.
Robb Pieper
What’s the point of Scrum? See if it knows that. What’s the whole point?
Jason Malmstadt
(Typing to ChatGPT) “What’s the point of Scrum?” Gonna give us on time and on budget? What’s it gonna give us?
(ChatGPT responding) "The main point of Scrum is to provide a framework for managing complex projects and products, particularly in the field of software development. It's designed to help teams work together more effectively and deliver high-quality products, on time and within budget. Scrum is based on the Agile philosophy emphasizing flexibility, adaptability, ability to respond quickly to changing requirements. The agile approach is based on the idea that it's difficult to predict exactly what a product will look like at the end of a project, and that requirements will change over time. Scrum provides a structure for accommodating this uncertainty embracing change so teams can deliver high-quality products and meet the needs of their customers and stakeholders. Scrum emphasizes transparency, inspection, and adaptation…"
(Hey, an empiricism mention, great!)
"…as a way of managing and improving a process. The goal is to create an environment where stakeholders, customers, and the development team can see the progress of the project and make adjustments as needed. Ultimately, the main objective of Scrum is to build and deliver valuable and useful products to the customers by providing a practical and efficient way of working, that empowers teams to deliver incrementally and continuously."
Alright, Robb give me your thoughts. What do you think?
Robb Pieper
Aw, man. Well, I was hoping to see something about providing done increments in a short timeframe, something like that. I’m not sure I saw that anywhere even kind of close. The point of Scrum is not to provide a framework. The point of Scrum is to get work done. It’s to adapt. That’s the point of Scrum. It is a framework. So, the goal is not to provide a framework, it is actually to get work done. Scrum is a framework. Yeah, that’s what I got. Greg, any thoughts? I don’t want to take ’em all here.
Gregory Crown
I feel like it’s really similar to the Scrum question. What is Scrum? So I’m not feeling too great about it. It did mention empiricism, so it gets a little +1 because I heart empiricism. But I’d be curious what ChatGPT thinks about creating a Definition of Done. Could it do it for my team? I actually ran that experiment the other day. I was surprised at the results. They weren’t awful. I wasn’t too mad.
Jason Malmstadt
I think I would agree, I would give this (a) passing (grade). I think it’s better than “What is Scrum?”, I think. I like this answer better. It mentions transparency, inspection, and adaptation which, empiricism. It still throws in that “on time, within budget”, so it’s really got a high connection there to this defined process control or traditional project management thinking. Which, again, I don’t mind things being on time and on budget, but that’s not a guarantee of success. It talks a lot about responding to change, to be flexible, adaptable, so I probably give this one another B/B- maybe?
Robb Pieper
I was gonna backpedal on one of my answers. So it said, the beginning parts, it said the main point of Scrum is to provide a framework. Now looking at the last paragraph, it’s talking about ultimately, the main objective of Scrum is to build and deliver valuable and useful products to the customers. So it kind of contradicts a little bit, part one, but I guess it ultimately does answer my question. So I’m gonna give it five more points.
Jason Malmstadt
Okay. Is this like “Whose Line (is it), Anyway?” where the points don’t really matter? Yeah, I think I would give this a B/B- maybe. Overall, does it get it right? Yeah, kind of in an inconsistent manner. Some of the terminology I think will throw some people off. If you didn’t know anything about Scrum, it’d give you a mostly accurate but I think incomplete and somewhat confusing picture of what Scrum is.
But I’m really interested in what Greg said there a minute ago, like what it would say about Definition of Done? What if we asked it to create a Definition of Done, what would that do? So maybe this can be the start of some more discussion. If you out there in YouTube Land if you’re watching this, if you’ve got some ideas you want us to throw at ChatGPT and see what our responses are, there’s a comment section below. So go ahead and throw some suggested questions, and maybe we’ll do this again soon. Anyway, that’s all I got. Thanks for watching.
