Alexander Pugh discusses why and when to use Robotic Process Automation (RPA). Host Jeremy Jung spoke with Pugh about interacting with systems without APIs like mainframes; the importance of having developers involved when building bots; the difficulty of automating GUIs using the DOM and COM, dealing with two factor authentication; why RPAs are a bridge and not a permanent solution; the fuzzy line between RPAs that communicate with APIs and custom applications; and why the future of RPAs will connect APIs instead of automating GUIs.
Enterprise RPA Solutions
Enterprise “Low Code/No Code” API Solutions
RPA and the OS
- Office primary interop assemblies
- Office Add-ins documentation
- Task Scheduler for developers
- The Component Object Model
- The Document Object Model
Transcript brought to you by IEEE Software
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected].
Jeremy Jung 00:00:16 Welcome to Software Engineering Radio. Today, I’m talking to Alexander Pugh. He’s a solutions architect with over five years of experience working on robotic process automation and cognitive services. Today, we’re going to focus on robotic process automation, Alexander, welcome to software engineering radio.
Alexander Pugh 00:00:33 Thank you, Jeremy. It’s really good to be here.
Jeremy Jung 00:00:35 So what does robotic process automation actually mean?
Alexander Pugh 00:00:39 It’s a very broad nebulous term. When we talk about robotic process automation, as a concept, we’re talking about automating things that humans do in the way that they do them. So that’s the robotic and automation that is done in the way a human does a thing. And then process is that thing that we’re automating. And then automation is saying, we’re turning this into an automation. We’re orchestrating this and automating this. And the best way to think about that in any other way is to think of a factory or a car assembly line. So initially when we went in and we automated a car or a factory automation line, what they did is essentially they replicated the process as a human, did it. So one day you had a human that would pick up a door and then put it on the car and bolt it on with their arms.
Alexander Pugh 00:01:39 And so the initial automations that we had on those factory lines were a robot arm that would pick up that door from the same place and put it on the car and bolt it on there. So the same can be said for robotic process automation. We’re essentially looking at these processes that humans do, and we’re replicating them with an automation that doesn’t in the same way and where we’re doing that is the operating system. So robotic process automation is essentially going in and automating the operating system to perform tasks the same way a human would do them in an operating system. So that’s RPA in a nutshell.
Jeremy Jung 00:02:19 So when you say you’re replicating something that a human would do doesn’t mean it has to go through some kind of gooey or some kind of user interface.
Alexander Pugh 00:02:27 That’s exactly right. Actually, when we were talking about RPA and we look at a process that we want to automate with RPA, we say, okay, let’s watch the human do it. Let’s record that. Let’s document the human process. And then let’s use the RPA tool to replicate that exactly in that way. So go double click on Chrome, launch that click in the URL line and send key in WW dot CNN com or what have you, or service now hit, enter, wait for it to load and then click, you know, where you want to, you know, fill out your ticket for service. Now send key in. So that’s exactly how an RPA solution at the most basic can be achieved now. And any software engineer knows if you sit there and look over someone’s shoulder and watch them use an operating system, you’ll say, well, there’s a lot of ways we can do this more efficiently without going over here, clicking that, you know, we can use a lot of services that the operating system provides in a programmatic way to achieve the same ends and RPA solutions can also do that.
Alexander Pugh 00:03:38 The real key is making sure that it is still achieving something that the human does and that if the RPA solution goes away, a human can still achieve it. So if you’re trying to replace a replicate a process with RPA, you don’t want to change that process so much so that a human can no longer achieve it as well. That’s something where if you get a very technical and very fluent software engineer, they lose sight of that because they say, oh, you know what? There’s no reason why we need to go open a browser and go to the, you know, the service now portal and type this in when I can just directly send information to their backend, which a human could not replicate. Right? So that’s kind of where the line gets fuzzy. How efficiently can we make this RPA salute?
Jeremy Jung 00:04:31 I think a question that a lot of people are probably having is a lot of applications have APIs now, but what you’re saying is that for it to be, I suppose, true RPA, it needs to be something that a user can do on their own and not something that the user can do by opening up dev tools or making a post to an end point.
Alexander Pugh 00:04:53 Yeah. And so it’s probably really important right now to talk about why RPA, right? Why would you do this when you could put on a server, a really good API ingestion point or trigger or a web hook that can do this stuff. So why would we ever pursue RPA? There’s a lot of good reasons for it. RPA is very, very enticing to the business. RPA solutions and tools are marketed as a low code, no code solution for the business to utilize, to solve their processes that may not be solved by an enterprise solution and the in between processes in a way you have a big enterprise finance solution that everyone uses for the finance needs of your business, but there are some things that it doesn’t provide for that. You have a person that’s doing a lot of, and the business says, okay, well, this thing this human is doing, this is really beneath their capability.
Alexander Pugh 00:05:55 We need to get a solution for it, but our enterprise solution just can’t account for it. So let’s get a RPA capability in here. We can build it ourselves. And then there we go. So there are many reasons to do that. Financial, it might not have the capability or the funding to actually build and solve the solution, or it’s at a scale that is too small to open up an it project to solve for. And so, you know, a team of five is just doing this and they’re doing it for, you know, 20 hours a week, which is a large, but in a big enterprise, that’s not really maybe worth building an enterprise solution for it. Or, and this is a big one. There are regulatory constraints and security constraints around being able to access this or communicate some data or information in a way that is non-human or programmatic.
Alexander Pugh 00:06:53 So that’s really where RPA is correctly and best applied and you’ll see it most often. So what we’re talking about there is in finance, in healthcare or in big companies where they’re dealing with a lot of user data or customer data in a way. So when we talk about finance and healthcare, there are a lot of regulatory constraints and security reasons why you would not enable a programmatic solution to operate on your systems. You know, it’s just too hard. We’re not going to expose our databases or our data to any other thing. It would take a huge enterprise project to build out that capability, secure that capability and ensure it’s going correctly. We just don’t have the money the time or the strength honestly, to afford for it. So they say, well, we already have a user pattern. We already allow users to talk to this information and communicate this information. Let’s get an RPA tool, which for all intents purposes will be acting as a user. And then it can just automate that process without us exposing to queries or any other thing in enterprise solution or programmatic solution. So that’s really why RPA, where, and watching you would apply. It is there’s just no capability at enterprise for one reason or another to solve for
Jeremy Jung 00:08:26 As software engineers. When we see this kind of problem, our first thought is, okay, let’s build this custom application or workflow. That’s going to talk to all these API APIs. And what it sounds like is in a lot of cases, there just isn’t the time there just isn’t the money to put in the effort to do that. And it also sounds like this is a way of being able to automate that and maybe introducing less risk because you’re going through the same security, the same workflow that people are doing currently. So, you know, you’re not going to get into things that they’re not supposed to be able to get into because all of that’s already put in place.
Alexander Pugh 00:09:12 Correct. And it’s an already accepted pattern and it’s kind of odd to apply that kind of very it software engineer term to a human user, but a human user is a pattern in software engineering. We have patterns that do this and that, and, you know, databases and not, and then the user journey or the user permissions and security and all that is a pattern. And that is accepted by default when you’re building these enterprise applications. Okay. What’s the user pattern. And so since that’s already established and well-known, and all the, hopefully, you know, walls are built around that to enable it to correctly do what it needs to do. It’s saying, okay, we’ve already established. Let’s just use that instead of, you know, building a programmatic solution where we have to go and find, do we already have an appropriate pattern to apply to it?
Alexander Pugh 00:10:04 Can we build it in safe way? And then can we support it all of a sudden, you know, we have the support teams that watch our Splunk dashboards and make sure nothing’s going down with our big enterprise application. And then you’re going to build a, another capability. Okay, where’s that support going to come from? And now we got to talk about change, access boards, user testing, and, uh, you know, UAT dev production environments and all that. So it becomes untenable depending on your organization to do that for things that might fall into a place that is, it doesn’t justify the scale that needs to be thrown upon it. But when we talk about something like API APIs and APIs exists for a lot of things, they don’t exist for everything. And a lot of times that’s for legacy databases, that’s for mainframe capability. And this is really where RPA shines and is correctly applied.
Alexander Pugh 00:11:04 And especially in big businesses are highly regulated businesses where they can’t upgrade to the newest thing, or they can’t throw something to the cloud. They have, you know, their mainframe systems or they have their database systems that have to exist for one reason or the other until there is the motivation and the money and the time to correctly migrate and solve for them. So until that day, again, there’s no API to do anything on a mainframe in this bank or whatnot. It’s like, well, okay, let’s just throw RPA on it. Let’s, you know, let’s have RPA do this thing. I, in the way that a human does it, but it can do it 24 7 and an example, or use cases, you work at a bank and there’s no way that InfoSec is going to let you query against this database with your users that have this account or your customers that have this no way in any organization at a bank, is InfoSec going to say, oh yeah, sure.
Alexander Pugh 00:12:07 Let me give you an O data query, you know, driver on, you know, and you can just set up your own SQL queries and do whatever they’re going to say no way. In fact, how did you find out about this database in the first place and who are you? How do we solve it? We go and say, okay, how does the user get in here? Well, they open up a mainframe emulator on their desktop, which shows them the mainframe. And then they go in, they click here and they put this number in there and then they look up this customer and then they switch this value to that value and they say, save. And it’s like, okay, cool. That RPA can do. And we can do that quite easily. And we don’t need to talk about APIs and we don’t need to talk about special access or doing queries that makes, you know, InfoSec very scared.
Alexander Pugh 00:12:50 You know, a great use case for that is, you know, a bank say they acquire a regional bank and they say, cool, you’re now part of our bank, but in your systems that are now going to be a part of our systems, you have users that have this value, whereas in our bank, that value is this value here. So now we have to go and change for 30,000 customers. This one field to make it line up with our systems. Traditionally, you would get a, you know, an extract, transform load tool, an ETL tool that kind of do that. But for 30,000 customers that might be below the threshold and this is banking. So it’s very regulated and you have to be very, very intentional about how you manipulate and move data around. So what do we have to do? Okay. We have to hire 10 contractors for six months, and literally what they’re going to do eight hours a day is go into the mainframe through the simulator and customer by customer.
Alexander Pugh 00:13:48 They’re going to go change this value and hit save. And they’re looking at an Excel spreadsheet that tells them what customer to go into. And that’s going to cost X amount of money and X, you know, for six months, or what we could do is just build a RPA solution, a bot, essentially that goes, and for each line of that Excel spreadsheet, it repeats this one process, open up mainframe emulator, navigate into the customer profile and then changes value, and then shut down and repeat. And it can do that in one week and can be built into that’s the dream use case for RPA, and that’s really kind of where it would shine.
Jeremy Jung 00:14:31 So it sounds like the best use case for it is an old system, a mainframe system in cobalt, maybe doesn’t have an API. And so it makes sense to rather than go, okay, how can we get directly into the database? Or, you know, how can we build on top of it? Let’s just go through the user interface that exists, but just automate that process and the, you know, the example you gave, it sounds very, very, well-defined, you’re going to log in and you’re going to put in maybe this ID, here’s the fields you want to get back and you’re going to save those and you didn’t have to make any real decisions, I suppose, in, in terms of, do I need to click this thing or this thing it’s always going to be the same path.
Alexander Pugh 00:15:21 Exactly. And that’s really, you need to be disciplined about your use cases and what those look like. And you can broadly say use case that I am going to accept has these features, and one of the best ways to do that and say it has to be a binary decision process, which means there is no dynamic or interpreted decision that needs to, or information that needs to be made. Exactly like that use case it’s very binary either is, or it isn’t you go in you journey into there and you change that one thing. And that’s it there’s no, oh, well, this information says this, which means then, then I have to go do this. Once you start getting in those processes, you’re going down a rabbit hole and it could get very shaky. And that introduces the extreme instability in what you’re trying to do.
Alexander Pugh 00:16:16 And also really expands your development time because you have to capture these processes, then you have to say, okay, tell me exactly what we need to build this bot to do. And for binary decision processes, that’s easy go in here, do this, but nine times out of 10, as you’re trying to address this and solution for it, you’ll find those uncertainties. You’ll find these things where the business says, oh, well, yeah, that happens. You know, one time that a 10 and this is what we need to do. And it’s like, well, that’s going to break the bot. You know, nine times out of 10, this lot is going to fall over. This is now where we start getting into the machine learning and AI realm. And why RPA is classified sometimes as a subset of the AI or machine learning field, or is a pattern within that field is because now that you have this bot or this software that enables you to do a human process, let’s enable that bot to now do decision-making processes where it can interpret something and then do something else.
Alexander Pugh 00:17:23 Because while we can just do a big FL tree to kind of address every capability, you’re never going to be able to do that. And also it’s, it’s just a really heavy, bad way to build things. So instead let’s throw in some machine learning capability where it just can understand what to do, and that’s, you know, that’s the next level of RPA application that is okay, we’ve got it. We’ve gone throughout organization. We found every kind of binary thing that can be replaced with an RPA bot. Okay. Now what are the ones that we said we couldn’t do? Because it had some of that decision-making that required too much of a dynamic intelligence behind it. And let’s see if we can address those now that we have this. And so that’s the 2.0 in RPA is addressing those non-binary paths. I would argue that especially in organizations that are big enough to justify bringing in an RPA solution to solve for their processes, they have enough binary processes, binary decision processes to keep them busy.
Alexander Pugh 00:18:33 Some people kind of get caught up in trying to right out of the gate, say, we need to throw some machine learning. We need to make these bots really capable instead of just saying, well, we’ve got plenty of work, just changing the binary processes or addressing those. Let’s just be disciplined and take that approach. I will say, towards RPA and bots, the best solution or the only solution. When you talk about building a bot is the one that you eventually turn off. So you can say, I built a bot that will go into our mainframe system and update this value. And that’s successful. I would argue that’s not successful when that bot is successful is when you can turn it off because there’s an enterprise solution that addresses it. And you don’t have to have this RPA bot that lives over here and does it instead, your enterprise capability now affords for it. And so that’s really, I think a successful bot or a successful RPA solution is you’ve been able to take away the pain point or that human process until it can be correctly addressed by your systems that everyone uses from
Jeremy Jung 00:19:46 The business perspective, you know, what are some of the limitations are long-term problems with leaving an RPA solution?
Alexander Pugh 00:19:56 That’s a good question from the business there isn’t, it’s solved for leaving it in places other than just servicing it and supporting it. There’s no real issue there, especially if it’s an internal system like a mainframe, you guys own that. If it changes, you’ll know it, if it changes, it’s probably being fixed or addressed. So there’s no problem. However, that’s not the only application for RPA. Let’s talk about another use case here. Your organization uses a bank and you don’t have an internal way to communicate it. Your user literally has to go to the bank’s website, log in and see information that the bank is saying, Hey, this is your stuff, right? The bank doesn’t have an API for that service, because that would be scary for the bank. They say, we don’t want to expose this to another service. So the human has to go in there, log in, look at maybe a PDF and download it and say, oh, okay.
Alexander Pugh 00:20:51 So that happens in a browser. So it’s a newer technology. This isn’t our mainframe built in 1980. It’s, you know, browser base, it’s in the internet and all that, but that’s still a valid RPA application, right? It’s a human process. There’s no API, there’s no easy programmatic way to solution for it. It would require the bank and your it team to get together and, you know, hate each other. And think about why this is so hard. So let’s just throw a bot on it. That’s going to go in, log in, download this thing from the bank’s website and then send it over to someone else. And it’s going to do that all day. Every day, that’s a valid application. And then tomorrow the bank changes its logo. And now my bot is confused. Stuff has shifted on the page. It doesn’t know where to click anymore.
Alexander Pugh 00:21:37 So you have to go in and update that bot because sure enough, that bank’s not going to send out an email to you and say, Hey, by the way, we’re upgrading our website in two weeks, it’s not going to happen. You’ll know after it’s happened. So that’s where you’re going to have to upgrade the bot. And that’s the indefinite use of RPA is going to have to keep until someone else decides to upgrade their systems and provide for a programmatic solution that is completely outside the capability of the organization to change. And so that’s where the business would say, we need this indefinitely. It’s not up to us. And so that is an indefinite solution that would be valid, right? You can keep that going for 10 years as long, I would say, you probably need to get a bank that maybe meets your business needs a little easier, but it’s valid. And that would be a good way for the business to say, yes, this needs to keep running forever until it doesn’t.
Jeremy Jung 00:22:28 You brought up the case of where the webpage changes and the bot doesn’t work anymore. Specifically, you’re, you’re giving the example of finance. And I feel like it would be basically catastrophic if the bot is moving money to somewhere, it shouldn’t be moving because the UI has moved around or the buttons not where it expects it to be. And I’m kind of curious what your experience has been with that sort of,
Alexander Pugh 00:22:53 You need to set organizational thresholds and say something this impacting or something that could go this wrong. It is not acceptable for us to solve with RPA, even though we could do it, it’s just not worth it. Some organizations say that anything that touches customer data, healthcare and banking specialists say, yeah, we have a human process where the human will go and issue refunds to a customer, and that could easily be done via RPA solution, but it’s fraught with what if it does something wrong? It’s literally going to impact someone somewhere, their monies or their security or something like that. That definitely should be part of your evaluation. And as an organization, you should set that up early and stick to it and say, Nope, this is outside our purview. Even we can do it. It has these things. So I guess the answer to that is you should get to that process, but, and now we’re going to talk about, I guess, the actual nuts and bolts of how RPA solutions work and how they can be made to not action upon stuff when it changes, or if it does.
Alexander Pugh 00:23:59 So RPA software by and large operates by exposing the operating system or the browsers underlying models and interpreting them. Right. So when we talk about something like a mainframe emulator, you have your RPA software on Microsoft windows. It’s going to use the comm the component operating model to see what is on the screen. What is on that emulator. It’s going to expose those objects to the software and say, you can pick these things and click on that and do that. When we’re talking about the browser, what the RPA software is looking at is not only the comm the component object model there, which is the browser itself. But then it’s also looking at the Dom the document object model. That is the webpage that is being served through the browser. And it’s exposing that and say, these are the things that you can touch or operate on.
Alexander Pugh 00:25:00 And so when you’re building your bots, what you want to make sure is that the uniqueness of the thing that you’re trying to access is something that is truly unique. And if it changes that one thing that the bot is looking for will not change. So let’s go back to the banking website, right? We go in and we launch the browser and the bot is sitting there waiting for the operating system to say, this process is running, which is what you want it to launch. And it is in this state, the bot says, okay, I’m expecting this kind of calm to exist. I see it does exist. It’s this process and it has this kind of name and cool Chrome is running. Okay, let’s go to this website. And after I’ve typed this in, I’m going to wait and look at the Dom and wait for it to return this expected webpage name, but they could change their webpage name, the title of it, right.
Alexander Pugh 00:26:00 They can say, one day can say, hello, welcome to this bank. And the next day it says bank website, all of a sudden your bot breaks it no longer is finding what it was told to expect. So you want to find something unique that will never change with that conceivably. And so you find that one thing on the Dom on the banking website, you know, this element or this tag said, okay, there’s no way they’re changing that. And so it says, cool. The page is loaded. Now click on this field, which is log in. Okay. You want to find something unique on that field that won’t change when they upgrade, you know, from bootstrap to this kind of, you know, UI framework, that’s all well, and good. That’s what we call the happy path. It’s doing this perfectly. Now. You needed to find what it should do when it doesn’t find these things, which is not keep going or find similar it’s it needs to fail fast and gracefully and pass on that failure to someone and not keep going.
Alexander Pugh 00:26:56 And that’s kind of how we prevent that scary use case where it’s like, okay, it’s gone in, it’s logged into the bank website. Now it’s, transactioning bad things to bad places that we didn’t program it for it. Well, you unfortunately did not specify in a detailed enough way what it needs to look for. And if it doesn’t find that it needs to break, instead of saying that this is close enough. And so in all things, software engineering, it’s that specificity, it’s that detail that you need to hook onto. And that’s also where, when we talk about this being a low-code no-code solutions that sometimes RPA is marketed to the business. It’s just so often not the case, because yes, it might provide a very user business friendly interface for you to build bots. But the knowledge you need to be able to ensure stability and accuracy to build the bots is a familiarity.
Alexander Pugh 00:28:39 It’s going to be inaccurate in its execution. This is a business friendly user-friendly non-technical tool. And I launch it and it says, what do you want to do? And it says, let me record what you’re going to do. And you say, cool. And then you go about you open up Chrome and you type in the browser, and then you click here, click there, hit send, and then you stop recording. The tool says, cool, this is what you’ve done. Well, I have yet to see a solution that is able to not need further direction or defining on that process. You still should need to go in there and say, okay, yeah, you recorded this correctly, but you know, you’re not interpreting correctly or as accurate as you need to that field that I clicked on. And if you know, anybody hits, you know, F 12 on their keyboard while they have Chrome open and they see how the Dom is built, especially if this is using kind of any kind of template webpage software, it’s going to have a lot of cruft in that HTML.
Alexander Pugh 00:29:38 So while yes, the recording did correctly see that you clicked on that input box. What it’s actually seen is that you actually clicked on the div, that is four levels scoped above it, or is the parent. And there are other things within that as well. And so the software could be correctly clicking on that later, but other things could be in there and you’re going to get some instability. So the human or the business bot builder, the robot is to say, yes, would need to say, okay, listen, we need to pare this down, but it’s even beyond that. There are concepts that you can’t get around when building bots that are unique to software engineering as a concept. And even though they’re very basic, it’s still sometimes hard for the business user to, they still have to learn that. And I’m talking concepts as simple as for loops or loops in general, where the business of course has knowledge of what we would call a loop, but they wouldn’t call it a loop and it’s not as accurately defined.
Alexander Pugh 00:30:44 So they have to learn that. And it’s not as easy as just saying, oh yeah, do a loop. And the business will say, well, what’s a loop. Like I know conceptually what a loop could be like a loop in my, when I’m tying my shoe. But when you’re talking about loop, that’s a very specific thing in software and what you can do. And when you shouldn’t do it, and that’s something that these, no matter how good your low-code no-code solution might be, it’s going to have to afford for that concept. And so a business user is still going to have to have some lower level capability to apply those concepts and have yet to see anybody be able to get around that in their RPA solutions.
Jeremy Jung 00:31:22 So in your experience, even though these vendors may sell it as being a tool that anybody can just sit down and use, but then you would want a developer to sit with them or see the result and try and figure out, okay, what do you really want this code to do? Not just sort of these broad strokes that you were hoping the tool was going to take care of.
Alexander Pugh 00:31:43 That’s exactly right. And that’s how every organization will come to that realization pretty quickly of the head of the game. Once they’ve said, okay, we need to have a really good COE structure to this robotic operating model where we can have a software engineering developer capability that sits with the business capability. And they can marry with each other, other businesses who may take these vendors at their word and say, it’s a low code meant for business. It just needs to make sure it’s on and accessible. And then our business people are just going to go in there and do this. They find out pretty quickly that they need some technical guidance to go in because they’re building unstable or inaccurate bots. And whether they come to that sooner or later, they always come to that. And they realize that, okay, there’s a technical capability. And this is not just RPA. This is the story of all low-code no-code solutions that have ever existed. It always comes around that while this is a great interface for doing that. And it’s very easy. He makes concepts easy. Every single time. There is a technical capability that needs to be afforded for
Jeremy Jung 00:32:58 Browser. You mentioned the Dom, which is how we typically interact with applications there. But for native applications, you briefly mentioned calm. And I was wondering when someone is writing, you know, a bot, what are the sorts of things they see, or what are the primitives they’re working with? Like, is there a name attached to each button, each text field thinks,
Alexander Pugh 00:33:22 So wouldn’t that be a great world to live in? So there’s not an, as we build things in the Dom, people get a lot better. We seen people are getting much better about using uniqueness when they build those things so that they can latch onto when things were built for the comm, you know, in.net for O S no one was like, oh yeah, we’re going to automate this. Or, you know, we need to make this so that this button here is going to be unique from that button over there, on the comm in care, different names. So, yeah, that is sometimes a big issue when you’re using an RPA solution, you say, okay, cool. Look at this calculator app. And okay, it’s showing me the component object model that this was built. That’s describing what is looking at, but none of these nodes have a name.
Alexander Pugh 00:34:12 They’re all, you know, node one node, 1.1 node two, or whatnot, or button is just button and there’s no uniqueness around it. And that is, you see a lot of that and legacy older software, either legacies, things built in 2005, 2010, you do see that. And that’s the difficulty at that point, you can still solve for this, but what you’re doing is you’re using send keys. So instead of saying, okay, RPA software, open up this application and then look for this, this object in the column and click on it. It can’t, there is no uniqueness. So what you say is just open up this software and then just hit tab three times. And that should get you to this one place that was not unique, but we know if you hit tab three times, it’s going to get there. Now that’s all well and good, but there’s so many things that could interfere with that and break it.
Alexander Pugh 00:35:09 And there’s no context for the bot to grab onto, to verify, okay, I am there. So anyone thing, you could have a pop-up, which essentially hijacks your send key, right? And so the bot yes, absolutely hit tab three times and it should be in that one place. It thinks it is, and it hits in her, but in between the first and second tab, a pop-up happened, and now it’s latched onto this other process at center. And all of a sudden Outlook’s opening bot doesn’t know that, but it’s still going on. And it’s going to enter in some financial information into oops and email that it launched because it thought hitting enter again, would do. So. That’s where you get that instability. There are other ways around it or other solutions. And this is where we get into the you’re using lower level software engineering solutioning, instead of doing it exactly how the user does it.
Alexander Pugh 00:36:01 When we’re talking about the operating system and windows, there are a ton of interrupt services and assemblies that a RPA solution can access. So instead of cracking open Excel, double-clicking on Excel workbook, waiting for it to load, and then reading the information and putting information in, you can use the, you know, the office 365 or whatnot, that interrupt service assembly and say, Hey, launch this workbook without the UI, showing it attached to that process that, you know, it is, and then just send to it, using that assembly, send information into it. And the human user can’t do that. It can’t manipulate stuff like that, but the bot can, and it it’s the same end as the human users trying. And it’s much more efficient and stable because the UI couldn’t afford for that kind of stability. So that would be a valid solution. But at that point, you’re really migrating into a software engineering, it developer solution of something that you were trying not to do that for. So why not just go and solve for it with an enterprise or programmatic solution in the first place? So that’s the balance
Jeremy Jung 00:37:15 Earlier you were talking about the RPA, it needs to be something that the person is able to do. And it sounds like in this case, I guess there still is a way for the person to do it. They can open up the Excel sheet and
Alexander Pugh 00:37:29 They can still achieve the same ends.
Jeremy Jung 00:37:30 Right? It’s just that the way the RPA tool is doing it
Alexander Pugh 00:37:34 Right, and more efficient and more stable, certainly, especially when we’re talking about Excel, you have an Excel with, you know, 200,000 lines, just opening that that’s your day, that’s going to Excel it, just going to take its time opening and visualizing that information for you. Whereas, you know, an RPA solution doesn’t even need to crack that open. It can just send data right directly to that workbook. And that’s a valid solution. And again, some of these processes, it might be just two people at your organization that are essentially doing it. So it’s, you know, you don’t really, it’s not at a threshold where you need an enterprise solution for it, but they’re spending 30 minutes of their day just waiting for that Excel workbook to open and then manipulating the data, saving it, and then, oh, their computer crashed. So you can do an RPA solution.
Alexander Pugh 00:38:23 It’s going to be, to essentially build for a more efficient way of doing it. And that would be using the programmatic solution, but you’re right. It is doing it in a way that a human could not achieve it. And that again is where the discipline and the organizational aspect of this comes in, where it’s saying, is that acceptable? Is it okay to have it do things in this way that are not human, but achieving the same ends. And if you’re not disciplined that creeps, and all of a sudden you have a RPA solution that is doing things in a way that the whole reason to bring that RPA solution is to not have something that did something like that. And that’s usually where the stuff falls apart. It all of a sudden perks their head up and says, wait, I have a lot of connections coming in from this one computer doing stuff very quickly with a, you know, a sequel query. And it’s like, what is going on? And so all of a sudden, someone built a bot to essentially do a programmatic connection. And it is like, you should not be who gave you those permissions, who did this shut down everything that is RPA here until we figure out what you guys went and did. So that’s the dance.
Jeremy Jung 00:39:35 It’s almost like there’s this hidden API or there’s this API that you’re not intended to use, but in the process of trying to automate this thing, you use it. And then if you’re, it is not aware of it, then things just kind of spiral out of control.
Alexander Pugh 00:39:49 Exactly. Right. So a use case of that would be, we need to get California tax information on alcohol sales. We need to see what each county taxes for alcohol to apply to something. And so today the human users, they go into the California tobacco, wildlife, whatever website, and they go look up stuff and okay, that’s very arduous. Let’s throw a bot on that. Let’s have a bot do that. Well, the bot developers, smart person knows their way around Google and they find out, well, California has an API for that. Instead of the bot cracking open Chrome, it’s just going to send this rest API call and it’s going to get information back and that’s awesome and accurate and way better than anything. But now all of a sudden ITCs connections going in and out, all of a sudden it’s doing very quickly and it’s getting information coming into your systems in a way that you did not know was going to be happening.
Alexander Pugh 00:40:46 And so while it is all well and good, it’s a good way for the people whose job it is to protect yourself or know about these things, to get very angry, rightly so that this is happening. That’s an organizational challenge and it’s an oversight challenge. And it’s a, it’s a developer challenge because what you’re getting into is the problems with having two technical people build these RPA bots, right? So on one hand we have business people who are told, Hey, just crack this thing open and build it. And it’s like, well, they don’t have enough technical fluency to actually build a stable bot because they’re just taking face value. On the other hand, you have software engineers or developers that are very technical that say, oh, this process. Yeah. Okay. I can build a bot for that. But what if I used, you know, these interrupt services, assemblies that Microsoft gives me and I can access it like that. And then I can send an API call over here. And while I’m at it, I’m going to S you know, I’m just going to spin up a server just on this one computer that can do this when the bot talks to it. And so you have the opposite problem. Now you have something that is just not at all RPA, it’s just using the tool to, you know, manipulate stuff, programmatically.
Jeremy Jung 00:42:02 So as a part of all, this is using the same credentials as a real user, right. You’re logging with a username and password. If the form requires something like two factor authentication or, you know, or something like that, like how does that work since it’s not an actual person,
Alexander Pugh 00:42:20 Right. So in a perfect world, you’re correct. A bot is a user. I know a lot of times you’ll hear, say, people will be like, oh, I have 20 RPA bots. What they’re usually saying is I have 20 automations that are being run for separate processes with one user’s credentials on a VDI. So you’re right. They are using a user’s credentials with the same permissions that any user that does that process has, that’s why it’s easy. But now we have these concepts, like two factor authentication, which every organization is using that should require something that exists outside of that bot users environment. And so how do you afford for that in a perfect world? It would be a service account, not a user account and service accounts are governed a little differently. A lot of times service accounts have much more stringent rules, but also allow for things like password resets, not a thing or two factor authentication is not a thing for those.
Alexander Pugh 00:43:18 So that would be the perfect solution, but now you’re dragging in it. So, you know, if you’re not structurally set up for that, that’s going to be a long slog. So what would want to do some people actually literally have a, we’ll have a business person that has their two factor auth for that bot user on their phone. And then just, you know, they’ll just go in and say, yeah, that’s me. That’s untenable. So sometimes what a lot of these, like Microsoft, for instance, allow you to do is to install a two factor authentication application on your desktop so that when you go to log in a website and says, Hey, type in your password. Cool, okay, give me that code. That’s on your two factor auth app. The bot can actually launch that app, copy the code and paste it in there and be on its way.
Alexander Pugh 00:44:08 But you’re right now, you’re having to afford for things that aren’t really part of the process you’re trying to automate. They are the incidentals that also happen. And so you have to build your bot to afford for those things and interpret, oh, I need to do two factor authentication. And a lot of times, especially if you have an entirely business focused RPA robotic model, they will forget about those things or find ways around them that the bot isn’t addressing, like having the authenticator app on their phone that’s, um, stuff that definitely needs to be addressed. And sometimes it’s only found at runtime like, oh, it’s asking for login. And when I developed it, I didn’t need to do that because I had, you know, the cookie that said you’re good for 30 days, but now, oh, no.
Jeremy Jung 00:44:56 Yeah. You could have two factor. You could have it asking you to check your email for a code. There could be a fraud warning. There’s like all sorts of, you know, failure cases that can happen.
Alexander Pugh 00:45:07 Exactly. And those things are, when we talk about third-party vendors, third-party provider vendors, like going back to the banking website, if you don’t tell them that you’re going to be using a bot to get their information or to interface with that website, you’re setting yourself up for a bad time, because they’re going to see that kind of at runtime behavior that is not possible at scale by user. So you run into that issue at runtime, but then you’re correct. There are other things that you might run into at runtime that are not, again, part of the process, the business didn’t think that that was part of the process. It’s just something they do that actually the bot has to afford for that’s part of the journey in building,
Jeremy Jung 00:45:57 When you’re building these thoughts, what are the types of tools that you used in the past? Are these commercial packages? Are these open source? Like what does that ecosystem?
Alexander Pugh 00:46:09 Yeah. In this space, we have three big ones, which is automation, anywhere, UI path and blue prism. Those are the RPA juggernauts providing this software to the companies that need it. And then you have smaller ones that are trying to get in there, or provide stuff in a little different way. And you even have now big juggernauts that are trying to provide for it, like Microsoft with something like power automate desktop. So all of these say three years ago, all of these softwares existed or all of these RPA solution softwares existed or operated in the same kind of way, where you would install it on your desktop. And it would provide you a studio to either record or define originally the process that was going to be automated on that desktop when you pushed play and they all kind of expose or operate in the same way, they would interpret the comm or the Dom that the operating system provided things like task scheduler have traditionally exposed.
Alexander Pugh 00:47:17 And they all kind of did that in the same way. Their value proposition in their software was the orchestration capability and the management of that. So I build a bot to do this. Jim over there built a bot to do that, okay, this RPA software, not only enabled you to define those processes, but what their real value was is they enable a place where I can say this needs to run at this time on this computer. And it needs to, you know, I need to be able to monitor it and it needs to return information and all that kind of orchestration capability. Now all of these RPA solutions actually exist in the, like everything else in the browser. So instead of installing, you know, the application and launching it and whatnot, and the orchestration capability being installed on another computer that looked at these computers and ran stuff on them.
Alexander Pugh 00:48:08 Now it’s, it’s all in the cloud as it were, and they are in the browser. So I go to wherever my RPA solution is in my browser. And then it says, okay, cool. You, you still need to install something on the desktop where you want the spot to run and it deploys it there. But I define and build my process in the provided browser studio. And then we’re going to give you a capability to orchestrate monitor and receive information on those things that you have, those bots that you have running, and then what they’re now providing as well as the ability to tie in other services to your bot so that it has expanded capability. So I’m using automation anywhere. And I built my bot and it’s going, and it’s doing this or that. And automation anywhere says, Hey, that’s cool. Wouldn’t you like your bot to be able to do OCR while we don’t have our own OCR engine, but you probably, as an enterprise do just use, you know, use your Kofax OCR engine or Hey, if you’re really, high-speed, why don’t you use your Azure cognitive services capability, we’ll tie it right into our software.
Alexander Pugh 00:49:21 And so when you’re building your bot, instead of just cracking open a PDF and send key control C send key control V to do stuff instead, we’ll use your OCR engine that you’ve already paid for to understand stuff. And so that’s how they expand what they’re offering into addressing more and more capabilities. But now we’re migrating into a territory where it’s like, well, things have API APIs, why even build a bot for them. You know, you can just build a program that uses the API and the user can drive this. And so that’s where people kind of get stuck. They’re using RPA on something that just as easily provides for a programmatic solution as opposed to an RPA solution, but because they’re in their RPA mode and they say, we can use a bot for everything. They don’t even stop and investigate and say, Hey, wouldn’t this be just as easy to generate a react app and let a user use this because it has an API. And it can just as easily monitor and support that because it’s in an Azure resource bucket, that’s where an organization needs to be clear eyed and say, okay, at this point, RPA is not the actual solution. We can do this just as easy over here. And let’s pursue this,
Alexander Pugh 00:50:52 Yeah. So not necessarily. So again, the business does not know what an IDE is. It’s a studio. Um, but you’re correct. It’s an ID. Whether we’re talking about blue prism or UiPath or automation, anywhere, they all have a different flavor of what that looks like and what they enable. Traditionally blue prism gave you a studio that was more shape based where you are using UML shapes to define or describe your process. And then there you are. Whereas automation anywhere traditionally used essentially lines or descriptors. So I say, Hey, I want to open this file. And your studio would just show a line that said open file. You know, although they do now, all of them have a shape based way to define your process. Go here. You know, here’s a circle which represents this. Let’s do that. Or a way for you to kind of more creatively define it in a, like a text-based way.
Jeremy Jung 00:52:56 Yeah. And I suppose at that point, then anybody who knows how to use the RPA tool, but isn’t familiar with that code. You wrote, they’re just, they can’t maintain it anymore.
Alexander Pugh 00:53:07 You have lost business continuity. And this goes back to our, it has to be replicable or close as close to the human process, as you can make it because that’s going to be the easiest to inherit and support. That’s one of the great about it. Whereas if you’re a low level, you know, programmer or dev who says, I can easily do this with a couple lines of, you know, dot net or, you know, TypeScript or whatever. And so the bot just starts up and executes. Well, unless someone that is just as proficient comes along later and says, this is why it’s breaking. You now have an unsupportable business solution. That’s bad Juju.
Jeremy Jung 00:53:50 You have the software engineers who they want to write code. Then you have the people who are either in business or an it that go. I don’t want to look at your code. I don’t want to have to maintain it. So it’s like you almost, if you’re a software engineer coming in, you almost have to fight that urge to write anything yourself and figure out, okay, what can I do with the toolset and only go to code event? Can’t do it any other way.
Alexander Pugh 00:54:17 That’s correct. And that’s the, it takes discipline more often than not, not as fun as writing the code where you’re like, I can do this. And this is really where the wheels come off is you went to the business, that business that I have this process, very simple. I need to do this. And you say, cool, I can do that. And then you’re sitting there writing code and you’re like, but you know what? I know what they really want to do. And I can write that now. And so you’ve changed the process. And while it is, and nine times out of 10, the business will be like, oh, that’s actually what we wanted. The human process was just as close as we could get nothing else, but you’re right. That’s exactly what we needed. Thank you nine times out of 10. They’ll love you for that.
Alexander Pugh 00:54:56 But now you own their process. Now you’re the one that defined it. You have to do the business continuity. You have to document it. And when it falls over, you have to pick it back up and you have to retrain. And unless you have an organizational capacity to say, okay, I’ve gone in and changed your process. I didn’t automate it. I changed it. Now I have to go in and tell you how I changed it and how you can do it. And so that, unless you have built your robotic operating model and your team to afford for that, your developer could be writing checks bigger than they can cash. Even though this is a better capability,
Jeremy Jung 00:55:34 Sort of touched on this before. And I think this is probably the last topic we’ll cover, but you’ve been saying how the end goals should be to not have to use the RPAs anymore. And I wonder if you have any advice for how to approach that process and what are some of the mistakes you’ve seen people make?
Alexander Pugh 00:55:56 I mean, the biggest mistake I’ve seen organizations make, I think is throwing the RPA solution. Now they’re building bots and they’re great bots, and they are creating that value. They’re enabling you to save money and also enabling your employees to go on and do better, more gratifying work. But then they say, that’s, it that’s as far as we’re going to think, instead of taking those savings and saying, this is for replacing this pain point, that we had to get a bot in the first place to do so. That’s a huge common mistake. It’s absolutely understandable if I’m a CEO or even, you know, the person in charge of enterprise transformation. It’s very easy for me to say, ha victory, here’s our money. Here’s our savings. I justified what we’ve done. Go have fun. Instead of saying, we need to squirrel this money away and give it to the people that are going to change the system.
Alexander Pugh 00:56:51 So that’s definitely one of the biggest things. The problem with that is that’s not realized until years later when they’re like, oh, we’re still supporting these bots. So it is upfront having a turnoff strategy. When can we turn this bot off? What is that going to look like? Does it have a roadmap that will eventually do that? And that I think is the best way. And that will define what kind of processes you do indeed build bots for is you go to it and say, listen, we’ve got a lot of these user processes, human processes that are doing this stuff. Is there anything on your roadmap that is going to then replace that? And they say, oh yeah, you know, in three years we’re actually going to be standing up our new thing. We’re going to be converting. And part of our analysis of the solution that we will eventually stand up will be, does it do these things?
Alexander Pugh 00:57:45 And so yes, in three years, you’re good. And you say, cool, those are the processes I’m going to automate. And we can shut those off. That’s your point of entry for these things not doing that leads to bots running and doing things even after there is a enterprise solution for that. And more often than not, I would say greater than five times out of 10, when we are evaluating a process to build a bot for easily five times out of 10, we say, well, no, actually you don’t even need to do this. Our enterprise application can do this. You just need retraining because your process is just old and no one knew you were doing this. And so they didn’t come in and tell you, Hey, you need to use this. So that’s really a lot of times what the issue is. And then after that, we go in and say, okay, now there’s no solution for this as a definitely a bot needs to do this. Let’s make sure number one, that there isn’t a solution on the horizon six months to a year, because otherwise we’re just going to waste time, but let’s make sure there is, or at least it or the people in charge are aware that this is something that needs to be replaced bot or no bot. And so let’s have an exit strategy. Let’s have a turnoff strategy
Jeremy Jung 00:58:56 When you have applications that are relatively modern, like you have a JIRA service now, you know, they must have some sort of API and it may just be that nobody has come in and told them, you just need to plug these applications to get.
Alexander Pugh 00:59:11 And so kind of what you’re hitting on surfacing is the future of RPA. Whereas everything we’re talking about is using a bot to essentially bridge a gap, moving data from here to there, that can’t be done, programmatically accessing something from here to there that can’t be done programmatically. So we use a bot to do it. That’s only going to exist for so long legacy can only be legacy for so long, although conceivably cause we had that big COBOL thing maybe longer than we we’d all like, but eventually these things will be upgraded. And so either the RPA market will get smaller because there’s less legacy out there. And so RPA as a tool and a solution will become much more targeted towards specific systems or we expand what RPA is and what it can afford for. And so that I think is more likely the case.
Alexander Pugh 01:00:08 And that’s the future where bots or automations aren’t necessary interpreting the comm and the Dom and saying, okay, click here do that. But rather you’re able to quickly build bots that utilize API APIs that are built in and friendly. And so what we’re talking about there is things like Appian or MuleSoft, which are these kind of API integrators are eventually going to be classified as RPA. They’re going to be within this realm. And I think where you’re seeing that at least surfaced or moving towards is really what Microsoft offering in that, where they have something called power automate, which essentially is just a very user-friendly way to access API APIs that they built or other people have built. So I want to go and I need to get information to service. Now service now has an API. Yeah, here, it can go in and build you a nice little app that does a little rest call to it or a rest API call to it gets information back, or you can go in and, you know, use Microsoft power automate and say, okay, I want to access service now.
Jeremy Jung 01:01:56 So moving from our PA’s being this thing, that’s going to click through website, click through a desktop application instead it’s maybe more of this higher level tool where the user will still get this. I forget the term you used, but this tool to build a workflow, right. A studio. Okay. And instead of saying, oh, I want this to click this button or fill in this form. It’ll be, I want to get this information from service now. And I want to send a message using that information to slack or to Twilio, or you’re basically talking directly to these different services and just telling it what you want and where it should go.
Alexander Pugh 01:02:36 That’s correct. So as you said, everything’s going to have an API, right? Seemingly everything has an API. And so instead of us, our RPA bots or solutions being UI focused, they’re going to be API focused where it doesn’t have to use the user interface. It’s going to use the other service. And again, the cool thing about API APIs in that way is that it’s not directly connecting to your data source. It’s the same as your UI for a user. It sits on top of it. It gets the request and it correctly interprets that. And does it the same thing with your UI where I say I click here and wherever it says, okay. Yeah, you’re allowed to do that. Go ahead. So that’s kind of the benefit to that, but to your point, the user experience for whether you’re using a UI or API to build an RPA bot, it’s going to be the same experience for the user. And then at this point, what we’re talking about, well, where’s the value offering or what is the value proposition of RPA and that’s orchestration and monitoring and data essentially, we’ll take
Jeremy Jung 01:03:45 Care of hosting these where you will take care of where they’re going to run, uh, giving you a dashboard
Alexander Pugh 01:03:51 That’s a hundred percent correct. It’s providing a view into that thing and letting the business say, I want to know code this. And I want to be able to just go in and understand that and say, oh, I do want to do that. I’m going to put these things together and it’s going to automate this business process that I hate, but as vital, and I’m going to save it, the RPA software enables you to say, oh, I saw they did that. And I see it’s running and everything’s okay in the world and I want to turn it on or off. And so it’s that seamless kind of capability that that’s what that will provide. And I think that’s really where it isn’t, but really where it’s going. It’ll be interesting to see when the RPA providers switched to that kind of language because currently and traditionally they’ve gone to business and said, we can build you bots or no, no, your users can build bots.
Alexander Pugh 01:04:40 And that’s the value proposition they can go in. And instead of writing an Excel where you had one very, very advanced user that was building macros into Excel with VBA and their unknown to it or anybody else instead, you know, built a bot for it. And so that’s their business proposition today. Instead, it’s going to shift and I’d be interested. See, when it shifts where they say, listen, we can provide you a view into those solutions and you can orchestrate them. And, oh, here’s the studio that enables people to build them. But really what you want to do is give that to your it and just say, Hey, we’re going to go over here and address business needs and build them. But don’t worry. You’ll be able to monitor them and at least say, yeah, okay, this is going.
Jeremy Jung 01:05:21 Yeah. And that’s a shift. It sounds like where RPA is currently, you were talking about how, when you’re configuring them to click on websites and gooeys, you really do still need someone with the software expertise to know what’s going on. But maybe when you move over to communicating with API APIs, maybe that won’t be as important. Maybe somebody who just knows the business process really can just use that studio and get what they need,
Alexander Pugh 01:05:50 Right? Because the API only enables you to do what it defined. So service now, which does have a robust API, it says you can do these things. The same as a user can only click a button that’s there that you’ve built and said they can click. And so that is you can’t go off the reservation as easy with that stuff, really what’s going to become prime or important is as no longer do I actually have an Oracle server physically in my location with a database. Instead I’m using Oracle’s cloud capability, which exists on their own thing. That’s where I’m getting data from. What becomes important about being able to monitor these is not necessarily like, oh, is it falling over? Is it breaking? It’s saying, what information are you sending or getting from these things that are not within our walled garden. And that’s really where it or the P InfoSec is going to be maybe the main orchestrator owner of RPA, because they’re going to be the ones to say, you can’t get that. You’re not allowed to get that information. It’s not necessarily that you can’t do it and you can’t do it in a dangerous way, but it’s rather, I don’t want you transporting that information or bringing it in. So that’s really, what’s going to change.
Jeremy Jung 01:07:05 I think that’s a good place to wrap it up, but is there anything we missed or anything else you want to plug before we go? No,
Alexander Pugh 01:07:13 This is a pretty comprehensive and I really
Jeremy Jung 01:07:15 Enjoyed it, Alex. Thanks for coming on the show.
Alexander Pugh 01:07:18 Thank you for having me. It’s
Jeremy Jung 01:07:19 Been a joy. This has been Jeremy Junge for software engineering radio. Thanks for listening.
SE Radio 01:07:26 Thanks for listening to se radio an educational program brought to you by either police software magazine or more about the podcast, including other episodes, visit our [email protected] to provide feedback. You can comment on each episode on the website or reach us on LinkedIn, Facebook, Twitter, or through our slack [email protected]. You can also email [email protected], this and all other episodes of se radio is licensed under creative commons license 2.5. Thanks for listening.
[End of Audio]