Search
Search
simon Bennetts
June 14, 2023

SE Radio 568: Simon Bennetts on OWASP Dynamic Application Security Testing Tool ZAP

Simon Bennetts, a distinguished engineer at Jit, discusses one of the flagship projects of OWASP: the Zed Attack Proxy (ZAP) open source security testing tool. As ZAP’s primary maintainer, Simon traces the tool’s origins and shares some anecdotes with SE Radio host Priyanka Raghavan on why there was a need for it. They take a deep dive into ZAP’s features and its ability to integrate with CI/CD, as well as shift security left. Bennetts also considers what it takes to build a successful open source project before spending time on ZAP’s ability to script to provide richer results. Finally, the conversation ends with some questions on ZAP’s future in this AI-powered world of bots.

Show Notes

SE Radio Episodes

References

Transcript

Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.

Priyanka Raghavan 00:00:16 Hello everyone, this is Priyanka Raghavan for Software Engineering Radio, and today we’ll be discussing OWASP ZAP, which stands for Zed Attack Proxy, a security testing tool with our guest, Simon Bennetts. Simon is a distinguished engineer at Jit and released ZAP in 2010. He has worked on nearly every part of ZAP’s code base and he’s given several talks and tutorial on ZAP, which are available on the official documentation page. You can probably hear a lot of excitement in my voice because I’m a big fan of his work. So it’s great to have you on the show and welcome.

Simon Bennetts 00:00:53 Thank you very much. Thank you for inviting me. It’s a pleasure to be here.

Priyanka Raghavan 00:00:57 We have done two episodes on OWASP and dynamic application security testing, episode 467 with Kim Carter on Dynamic Application Security Testing and episode 514 with Vandana Verma on OWASP Top 10. So maybe we can start right at the top, Simon. And the first question I wanted to ask you is, what is dynamic application security testing, which we can keep talking about and does ZAP fall under this category?

Simon Bennetts 00:01:36 So yes, ZAP is a dynamic application security testing tool, otherwise known as DAST. And there’s quite a few different types of security testing. As you well know. There’s static security testing and that’s where you’d look at the source code and you can see certain, there’s types of vulnerabilities you can find that way. ZAP doesn’t work in that way. ZAP doesn’t look at the source code at all. It actually looks at the running application. So this is, I wouldn’t say that DAST is better than SAST or vice versa, they’re just different ways of approaching the same thing. What we’re trying to do is find vulnerables in applications. ZAP is focused on web applications and what ZAP does, it interacts with the application via http, https, web sockets, all those web technologies. So, ZAP does attack your application if you tell it to — it’ll only do what you tell it to do — but in some ways you can think of it like it’s trying to do the same things as a malicious attacker. So somebody who knows about web vulnerabilities and tries to attack your application. Now ZAP tries not to do any damage, but I have taken out many websites in the past, unintentionally. So we don’t try and delete data from databases, but ZAP can put quite a lot of strain on websites, particularly if they’re not used to a lot of traffic. So it can be tricky. So you should, we should stress, you should only use ZAP on applications that you have permission to test or that you yourself own.

Priyanka Raghavan 00:03:04 Yeah, I think that makes a lot of sense. And also a good warning to our users that definitely try to make sure that you have permissions to test what you’re producing. Maybe as a software engineer it’s probably okay to use ZAP, but then make sure it’s only in the dev environment. Okay. The other thing I wanted to ask you was, I was reading somewhere in one of these blogs that said that ZAP was actually born out of a need for testing an application that you were working on. So can you tell us a little bit about that?

Simon Bennetts 00:03:36 Sure. So my background is software development. So this is in 2009. I was a developer and team lead. It was a small team, and we were developing an online application for a FTSE 100 company in the UK. So that’s one of the top 100 companies. And we knew it was security critical. So we planned around that and we implemented the service and a couple of weeks before it was supposed to go live, we got the penetration testers in just to — the whole idea was this was as far as I was concerned, yeah, it was a tick in the boxing just to prove we’d done everything right. It didn’t quite work out that way. So, got a couple of guys in and explained everything about the service because I knew they’re on our side. We wanted to find any vulnerabilities before it went live, obviously. Put them in a room, explained everything and let them get on with it.

Simon Bennetts 00:04:27 And went back an hour later just to see if I could explain anything, if there’s anything they didn’t understand. And I still remember walking into that room and seeing one of the pen testers logged into the admin console with my credentials. They shouldn’t have had those, they had, he had got super user access to the service. That was a problem. It was actually worse than that because although it wasn’t actually a vulnerability in the service I developed or my team developed, that actually cracked the single sign-on service for the whole company. This is a FTSE 100 company. They had cracked the single sign-on service in one hour. This was a bit of a wakeup call. It’s at that point I thought, okay, this week is not going to go the way I hoped. And at the end of the week, it seemed like a car crash.

Simon Bennetts 00:05:13 The report seemed awful. I now know it wasn’t as bad as it appeared, and I’ve delivered worse reports myself now. But it did not feel good at the time. So, I just kind of had to, I felt particularly bad and I decided to take stock. I was a developer, and I knew I was good at developing web services. I could develop services that did what they were supposed to do. They were performant; they were maintainable. They clearly weren’t secure enough. So, it was time for me to learn about security. One of the pen testers had told me about OWASP, which I’m afraid to say at that time I hadn’t heard of. And if any of your listeners haven’t heard of OWASP, it’s the open web application security project. Actually, it’s open worldwide application security project. I don’t think it’s just web anymore.

Simon Bennetts 00:05:58 So I hadn’t heard of OWASP, looked at OWASP and they’ve got something called the top 10 most common risks to web applications. So that covers things like your cross-site scripting and SQL injection. So I read all that, but when it comes down to it, I’m a developer and I like playing with things. I don’t learn as well from reading stuff. So I decided what I was going to do is I would find some tools to help me, and I’m a big fan of open-source, so I decided to go down the open-source route, and I didn’t really want to go to management and say, you’ve got to spend loads of money on tools. And I kind of wanted to, I wanted a tool that allowed me to do several things. One, I wanted to actually use it to run on my own software every night, so I wouldn’t get embarrassed by the pen testers again.

Simon Bennetts 00:06:42 But I also wanted to learn from it. So I didn’t just want something that you pressed a button and it magically did everything. I wanted to see what was going on underneath. So I wanted a tool I could learn from, and I’ve always had side projects and I thought, well maybe, this would be a good time to get an open-source. Maybe it’s an open-source web security project I could get involved in. So I had a look round, looking for a maintained open-source web security tool. And at that time there were none, absolutely zero. And that felt wrong to me, but it is what it was, what it was. So what I did was I found some tools that had been, that were no longer being maintained. There was an tool called WebScarab, which I didn’t really get on with. It was quite complicated and didn’t work the way, it just didn’t seem to gel with me. But there was another tool called Paris Proxy, which is quite nice, quite simple. I started playing around with that and to cut a long story shorter. I ended up forking it and creating ZAP from that fork.

Priyanka Raghavan 00:07:40 Wow, that’s a great story. Which leads to my next question. So was the tool then built with an audience that was only for developers, or is it OK also for pen testers?

Simon Bennetts 00:07:52 Yeah, so I mean when I released it, I was definitely much, a developer and didn’t consider myself a security person. So there was an old security list called Bug Track. So I posted a message on there saying that I was releasing this tool, it was a fork of Paris proxy. It was really, the tagline was actually ‘the security tool for developers.’ So I was definitely going for that. But I said maybe some security professionals might find it useful as well. But I really, I didn’t feel I had the cheek to say it was a suitable tool for security professionals when I wasn’t one myself. We kept that tagline for quite a while until I started going to OWASP events and pen testers kept on coming up to me and saying, hey, it’s not just for developers; we use it as well. So after a few years we kind of decided we had to drop that tagline, and we’ve kind of realized over the years that a lot of security people use ZAP, and I suppose in the years I probably have to claim I’m a security person now myself as well, as well as being a developer.

Priyanka Raghavan 00:08:51 Okay, great. So the audience is both pen testers as well as developers or anybody with an interest in testing.

Simon Bennetts 00:08:59 Exactly, and we’ve made things a bit difficult for ourselves really because our audience goes from people who are, who know nothing about security. They could be developers, they could be students, doesn’t really matter if you’re technical but interested in security, then web security then ZAP is a tool for you. But it goes all the way up to hardcore pen testers who know exactly what they want and probably use multiple, a whole range of tools. But ZAP will be one of them, and they need to understand the strengths and weaknesses of each tool and use them as appropriate. So it’s difficult to keep everyone happy. We can’t, but we do our best.

Priyanka Raghavan 00:09:31 That’s actually true, the line that you just said, making everyone happy, that’s really tough. But one thing about ZAP is what is the thing that you think that differentiates you from the other tools out there in the market, and why are you still open-source?

Simon Bennetts 00:09:45 Oh, good questions. So I mean one of the biggest things I think is being open-source. There are a few other open-source web security tools, but nothing quite like ZAP. So, we’re open-source, we’re community-based, so we want anyone to be able to get involved. So that for me is a key differentiator. But we know, and we know we do compete with commercial tools, and some of those commercials also have a lot of people working on them — a lot more than we do. But I still think ZAP has some significant strengths. I think in automation we’re probably the best DAST tool out there, and our API is incredible. It’s you can do pretty much anything via the API, and our scripting capabilities are second to none as well. So you can essentially rewrite ZAP on the fly virtually. We know that some power users make heavy use of the scripting features. Because ZAP is open-source, we’ve got nothing to hide. So the scripting interface can access all of the ZAP classes, it can access all of the data structures. We consider that the code belongs to community and the data belongs to whoever’s using ZAP. So you should be able to get hold of anything you want to and do whatever you like with ZAP, and if you can’t let us know and we’ll make sure you can.

Priyanka Raghavan 00:10:59 Okay. I think that brings me on to my one last question before we jump into ZAP of that, the thing with, the point that you made about APIs and extensibility that makes it easier to, for lack of a better way of explaining: shifting left, which I don’t like now nowadays, but then anyway, shifting left security. Do you have like a story which might say that somebody used ZAP and then, they went from this usual waterfall model that you’ve talked about in, was it 2009 that you said to now using ZAP and everything is like they tested immediately because of all the hooks that you have?

Simon Bennetts 00:11:33 I wish I had more stories. One of the problems is, as an open-source tool, anyone can download it and use it and they don’t have to tell us. So not many people tell us about how they use ZAP. We do have a few user stories, success stories on the website, but if anyone out there has got stories about how you use ZAP, would love to hear them. I did hear, talking on the extensibility recently, I heard from someone on the OWASP board about a very large bank in the United States who tested all of the DAST tools out there, all the main ones, including all the main commercial tools, and it couldn’t get any of the commercial tools working with their application because of some technical reasons. And ZAP was the only one they could actually get working, and ZAP didn’t work out of the box, but it came down to scripting: because we’ve got such an extensive scripting capability, they were able to actually change ZAP Script ZAP so that it could handle their particular case. So that’s one of the biggest banks in America can only use ZAP to scan its web applications at the moment.

Priyanka Raghavan 00:12:38 That’s impressive to hear. And I guess the question I wanted to ask you now is like considering its open-source, how do you have this shared vision and team cohesiveness and to produce something that was constantly, like every time you look at the ZAP marketplace, there are frequent updates. How do you manage that?

Simon Bennetts 00:12:57 With difficulty? So we have a small core team. There’s four of us. I think it’s, we probably, it’s only the equivalent of two people working full-time on ZAP really. But, we’ve got a lot of other contributors. So, and one thing we’re trying to do now is trying to work out how to actually resource ZAP better because we need more people working on ZAP. So we’re trying to get, we know a lot of commercial companies are actually wrapping ZAP in commercial products, so we’d love them to contribute back more. Some of them already do, but the company I work for, Jit, they’re sponsoring my work on ZAP, which is brilliant. And we’d love more of the companies that use ZAP, more of them to actually do something similar. So, we’d love people to be sponsored to work on ZAP. We’d love to raise more money so we could pay people to work on ZAP. There has been some controversy within OWASP. We’re trying to convince OWASP to make it easier to focus more on funding projects. But when it comes down to it, I think raising money has never been one of our strengths, I’m afraid.

Priyanka Raghavan 00:13:57 Okay. So more than the technical problems, I think it’s the, would it be fair to say that funding is one of the challenges with maintaining an open-source project?

Simon Bennetts 00:14:06 Definitely. If you need a significant number of, a significant amount of effort, it’s something you can’t do in your spare time. I started off doing ZAP in my spare time, and I wasn’t able to get very far. Luckily, Mozilla came along and sponsored my work, and now Jit is sponsoring it. But we really need people being able to spend a decent amount of time. A couple of the ZAP core team, most of rest of the ZAP core team do it in their spare time, which is incredible. But we need people to be able to dedicate more of their time to ZAP.

Priyanka Raghavan 00:14:37 One last question I wanted to know is how did ZAP become like a flagship project of OWASP? Is it because of the kind of contributions that is there on Jit? How does that work?

Simon Bennetts 00:14:49 Oh, good question. I mean I think, I’m not sure what the process was at the time. I know what the process is now because I’m actually on the OWASP project committee. So the idea is that projects can request to go up, I think it’s incubator, then labs, then production is that the labels we use. Flagship is something different. So flagship is something that is much more significant to OWASP. It’s not just saying it’s a great project, it’s also saying it’s a great project but it’s key to the direction of OWASP. So I think that is a board decision to decide to actually make a project flagship. I wasn’t involved in the decision at the time, but I think because ZAP stuck around so long and because ZAP became so well-known and so widely used, it actually has such a very beneficial effect on OWASP.

Priyanka Raghavan 00:15:40 That’s good to hear. So now actually I’d like to switch gears and go into little bit on the tool itself for our audience, which is predominantly a lot of software engineers, but right now with someone like me who’s coming a little bit with a security background, we also have that group. So let’s talk a little bit on the two version of ZAP that you have. You have like a desktop version and also a ZAP daemon, serialized. So can you tell us how that came about?

Simon Bennetts 00:16:09 Sure. So I, it’s actually only one version of ZAP. You can run it in different ways. So, originally ZAP was just a desktop tool, and that’s because that’s what Paris was — Paris Proxy, which I forked originally — but my whole idea I think as I initially said, what I wanted originally was a tool I could automate to test my own applications. So having a command line version was one of the things I wanted to work on pretty early on. So, the command line version got implemented quite early on, but since then we’ve found that people there are lots of different use cases for the way people want to run ZAP. So we’ve got a quite \wide range of options now. We’ve still got the desktop so you can have this swing UI that either Java swing UI that you can interact with, and we still recommend that’s a good way to learn about ZAP because you can then see what’s going on — if you want to debug in, I think it’s much easier.

Simon Bennetts 00:17:02 You can see all the requests and responses you can play with things interactively. So the desktop version is still very important. We have a daemon mode, as you mentioned, so we can put ZAP into the background, no UI, and then we’ve got a very thorough API which allows you to do nearly as much as you can do from the desktop GIU — not quite, but nearly. Then we have the automation framework, and the automation framework is a bit different because it allows you to control ZAP from one YAML file. So it has a series of jobs and those jobs can do things like running the spiders, running the active scanner, importing API definitions — kind of the things we expect we think people would want to do most frequently in automation. So you can create this YAML file, and you can actually, that works in both the desktop and from the command line.

Simon Bennetts 00:17:53 So you can play around with it, get it working on the desktop and then save it and run it from the command line. We also have some what we call package scans, and they basically run ZAP in specific ways where actually those are actually Python scripts, which were migrating to the automation framework but they only run in Docker. So we have Docker images, and that’s where the package scans live. Now the automation framework will run in Docker one from the command line. It isn’t dependent on the container. So the automation framework is a bit more flexible in that way. And we also have the heads-up display, which is another way of running ZAP, and this is where we actually add controls to your field of vision. So we actually decorate the browser with ZAP controls and information so you can see what’s going on while you’re focusing on your application and how it actually works, both in daemon and desktop mode as well. So we have all of those options.

Priyanka Raghavan 00:18:51 Yeah, I’m going to add some show notes to the HUD and so that people can actually look at their visual. But one of the things I was curious when I was working on the HUD was, where are you doing it so that? Like, in the future maybe you would blow up this onto a big screen and have one of those virtual glasses and then you go and point on something. I mean, I’m just asking, just curious. Because it almost looks like you would do something like that.

Simon Bennetts 00:19:16 I am actually a big fan of virtual reality, so I’ve got my Quest2 behind me here, and I have actually used ZAP in virtual reality, but I think that was just connecting to a desktop. So one of the problems with the browsers in VR at the moment, you don’t get as much control over them and you can’t, there’s not an easy way to proxy them through security tools like ZAP. So it is something I’d love to have a play with, but technically it’s quite tricky, and I mean they’re often solutions to these things but they can take a while and it’s probably not high enough anywhere near high enough of my priority list for me to be able to play that. But yeah, I would love to be able to use ZAP in VR and have the controls floating around the browser there. That would be brilliant. Yeah.

Priyanka Raghavan 00:20:02 So maybe any listeners on the show who have graphics experience should probably contribute to that then?

Simon Bennetts 00:20:08 Yeah, definitely. And I’ve, I’ve played around a bit with web VR as well and I was wondering whether we could actually have some information from ZAP Can is itself a web server. So we do have web interface, web VR interface to ZAP, but integrating that with a browser and actually seeing what the user sees is technically quite tricky.

Priyanka Raghavan 00:20:29 Okay. We had actually done an episode 474 on fast testing, not done by me but another host, and they talked a lot about how fuzzing is very important for doing security testing or even, normal testing. And I do see that we have a fuzzer in ZAP. So can you talk a little bit about this fuzzing support that ZAP provides?

Simon Bennetts 00:20:52 Sure. So one of the things with web application security testing is it’s always a balance. So you can throw random stuff or loads and loads of potentially bad things at an application and see what happens. But it takes a long time, particularly if you think that web applications typically have a lot of things you can attack because you have URL parameters, you have form parameters, you have headers, you’ve got web sockets; there’s loads of potential things. And so people often say that DAST tools like ZAP take a long time, and they can take a long time because there’s so much to do. So what we typically do with ZAP is we have what we call scan rules and we have passive scan rules which just look at things and spot potential problems without actually interacting with the application. Then we have the active scanner and the active scan rules, and these will actually attack the application.

Simon Bennetts 00:21:48 And what we do is we quite fine-grained control over what those rules can attack. So you can tune it to be what you want, but if you turn everything on then ZAP will attack quite a few things. So, in general, what we try and do is we try and do very targeted attacks. So for example, for cross-site scripting, what we’ll try and do is inject a safe token — send a safe token across, and see if it’s reflected in the website. If it’s reflected there, then we’ve got more of a chance. So we’ll then, we’ll look at the context within the HTML where it’s reflected and try and break out of those contexts so we can actually run some JavaScript. So we can actually do, we can kind of focus quite quickly onto potential problems, and we try not to make too many requests that aren’t actually useful or don’t appear to be useful from the automated side.

Simon Bennetts 00:22:40 So we don’t call what we do what with our active scanner a fuzzer because it’s very targeted with what it does. However, we do have a fuzzer, as you mentioned, and this is for us, it’s a very manual process because if we know how to detect potential vulnerabilities then we put those rules into, we codify that as part of the scan rules, but we know we can’t cope with everything and applications are very specific, and a security professional might well be looking for some very strange behavior, some unusual things. So what we have is a fuzzer and with that a pen tester can select one particular request and then they can select exactly which characters they want to change. And there’s a whole series of rules so you can, you can specify exactly what the payloads are, you can generate payloads, you can get payloads from a file, you can write scripts for payloads, you can put in processes. So you process every message, every payload. It is very, very flexible, but it is very much a manual process. So, and that’s one thing I mentioned, the API allows you to do most things, it doesn’t allow you to do fuzzing at the moment; we do plan to add the API to fuzzing, but it’s complicated, and it’s one of the harder ones to automate.

Priyanka Raghavan 00:23:54 Okay. And there was a lot of stuff you told us there. So let me just ask you one more question to summarize what you said. So you said that you do have a one piece, which is of course the fuzzer, which you can do for more sort of testing from a pen tester’s perspective and check particular parts with some kind of complicated inputs maybe, whereas you have the active scanner which you said, which also does this thing for you like for free like so yeah, if I didn’t know how to use the fuzzer, I would go in and use this active scan.

Simon Bennetts 00:24:26 Exactly, yeah. So what we’re trying to do is make sure that the ZAP is as easy to use as possible for people who are new to security. It’s tricky because a lot of security concepts are non-intuitive. We do weird stuff in security so it is a little bit tricky, but we try to make it as easy as possible. We try to make sure that newcomers can get started, but there’s hidden depths where you can do a lot more with ZAP as you learn.

Priyanka Raghavan 00:24:53 Okay. And one of the other things I wanted to ask, talking a little bit about the active and the passive scan, I remember once we had a story where I had actually spoken to one of the developers on my team and asked them to try using ZAP, and I think they had just blindly used it on the app and I think it just wiped out the entire dev database. So, like they were coming to me like, Priyanka what happened? You asked us to use this and it like just deleted all our things. The thing that I wanted to ask is that there are two options right there. Is there an option to do something like passive testing the system?

Simon Bennetts 00:25:28 Yeah, so I mean ZAP does what you tell it to do. So if you don’t tell it to attack anything, it won’t attack anything, but we are also aware that people can be nervous with security tools. So we have what we call modes, and we have a safe mode and if you put ZAP in safe mode, it won’t allow you to do any bad things. So ZAP can actually be very useful for testing things on production sites. You might actually want to see what requests and responses are being made. You might, I mean, I’ve used that for debugging before. It’s particularly useful when you’re not sure what requests being made by JavaScript libraries or the like, so you can put ZAP in safe mode and it won’t do anything bad at all.

Simon Bennetts 00:26:11 Then we have protected mode and protected mode. And it’s just the same as safe mode, unless you actually tell ZAP you want to attack something. So ZAP has this idea of contexts, and contexts can mean different things but basically you could think of it like an application. So you add your application to a context, you say it’s in scope, then ZAP will allow you to attack things in that context, that application, but won’t allow you to attack anything else. So protected mode is probably a good one for a lot of people. We have the standard mode, which allows you to do everything, and I’m afraid that’s what I use all the time but obviously I know how ZAP works so I know not to attack things when I shouldn’t do. We actually have an attack mode, as well. And that is where the way we usually recommend to use ZAP is you explore the application first, then you start the active scanner.

Simon Bennetts 00:26:58 We actually have this option where we put in attack mode and as soon as you actually say something is in scope, then ZAP will attack it and essentially it follows you around. So as you discover more things, ZAP will attack it. So if say you’ve got a large application and you want to focus on one particular part of it, if you use the spider, one of the spiders, it’ll be very difficult to restrict ZAP to that functionality. Whereas you can put it in, if you explore the application manually and put it in attack mode, then you can basically just invoke whatever functionality you want to test from the browser, and ZAP will only attack that functionality.

Priyanka Raghavan 00:27:36 Okay. Good to know. And once you are finished with the scan, what are the results that one would get? Does it just provide a list of exceptions with endpoints and severities?

Simon Bennetts 00:27:48 So we provide a lot of information, as much information as we can. And so, that will include obviously the vulnerability, we’ll show you the request and response; if you’ve got any evidence it’ll be there. The payload we used, there’ll be a solution in there, there’ll be a description, there’ll be links to other resources. We try and provide as much information as possible. We tag things like the different, OWASP top 10 categories in web application security guide categories. We have a reporting add-on which allows you to generate reports in a whole range of formats. So they could be HTML, PDF, JSON, XML, and that’s actually very extensible. So we use a Java library called Thymeleaf. So you can create your own reports; you don’t need to be a programmer. We’ve got all of our templates are written in Thymeleaf so you can actually, it’s just a kind of markup language really. So you can create your own reports, and we know people have done that, but we do have another add-on which integrates with bug trackers as well. So you can actually go down that route as well if you want to, if you want to automatically update a bug tracker.

Priyanka Raghavan 00:28:54 There’s also this thing in GitHub right now that’s called this security tab, right? Which has this with GitHub advanced security where you can see all, I think there’s a format called SARIF. So is that also some, oh you or should a potential person use this Thymeleaf. Is that what you said, Thymeleaf?

Simon Bennetts 00:29:15 Yes Thymeleaf. Let me just check. I think we have the SARIF format. I’m just going to look on the website now just to see. So we actually have, so all of the add-ons and all of the alerts from the website and yes, we have a SARIF JSON report now as well.

Priyanka Raghavan 00:29:30 Okay.

Simon Bennetts 00:29:31 One thing I forgot to mention is, you can actually run ZAP in GitHub Actions and the GitHub Actions will, I think they raise GitHub issues rather than the security alerts at the moment. But you can raise issues and track your potential vulnerabilities that way as well.

Priyanka Raghavan 00:29:46 Okay, okay that’s good to know. And picking up on that, I think a few years back I remember I used to work for an organization that was using Jenkins and then for CI, and then I obviously integrated ZAP for that, and then after some time they went onto another tool called Argo. And yeah, again that was very easy for me to integrate. One of the exercise that since I did these two exercises, of course I wrote a blog about it and stuff and I found that it was very easy to integrate almost any sort of CI tool with ZAP. So when you’re building a system, is that what you’re always thinking about, the ease of integration with like absolutely anything? Like is that a bit mind boggling when you’re designing something? Cause there’s so much out there.

Simon Bennetts 00:30:24 Yeah, that’s actually very important to us. So, I mean obviously we think ZAP is important, but we are very aware that ZAP is not an overall solution. It is not doing everything for everyone, and we couldn’t do that. It’s better if you have tools focused on particular things. So ZAP is focused on DAST scanning, and we know that people will want to integrate ZAP findings, they’ll want to interact with ZAP. You might want to feed information from one tool into ZAP. So having ZAP as a good citizen is very important. So we always think about ways that — we try to think about different ways tools can interact with ZAP without being a, thinking about specific tools. We want to make sure ZAP is easy to run from the command line, you can access as much functionality for the API as possible, and that we allow ZAP data to be accessed in as many ways as possible. So playing well with other tools, whether they’re commercial, open-source, or whatever, or custom ones, that people write for specific purposes, that is a very important thing. That’s something we always bear in mind. So if someone adds a new feature and they don’t add an API or so an option like that, then that would be picked up in the review and be like, oh could you put this in because we know that’s important to a lot of people.

Priyanka Raghavan 00:31:40 And that brings me to another point, there was a controversial topic a few days back where they talked about when you use any of these clean code concepts and you have a lot of modifiability or extensibility, then there’s also something that affects your performance. And I think the person who’s written it was talking more in terms of performance. So that’s one thing that just struck me while I was chatting with you now, how does this affect your ZAP performance? How fast would it be to run a test if it’s a part of your CI setup?

Simon Bennetts 00:32:10 I mean, that is one problem with DAST tools because there’s so much, if you talk about the whole application, there’s so much to test, typically. So tools like ZAP, you often think a lot of people think they can take some time to run, and if you’re testing the whole application, that’s very true. When I was working at Mozilla, that’s why I developed what we call the baseline scan where we basically do a very quick crawl of the application and just passively scan it. That typically finishes in a couple of minutes. So that can be very quick. But it’s also, ZAP is very, very flexible. So ZAP doesn’t understand source code, but if your static analyzer understands source code and can map that source code to endpoints, then you could get your CICD system to tell ZAP to only attack the endpoints that are affected.

Simon Bennetts 00:33:00 If you can do that, then ZAP will go very fast. If we’re attacking a couple of URLs, even with all of the scan rules enabled, it will be quick. So it’s very much a test question of how you drive ZAP. And that’s something unfortunately is outside of our control because ZAP isn’t a static source analyzer and is never going to be; there’s too many different types of dev stacks out there. But if your static analyzer with static code can tell ZAP which URLs are affected, then you can get ZAP to actually just attack those URLs. So ZAP is very, very flexible. It’s just a question of how you drive it.

Priyanka Raghavan 00:33:35 Okay, so if you were to run it as a part of our CI process, then maybe you have to use one of those baseline scans to do something under a minute?

Simon Bennetts 00:33:45 Yeah, unless you can actually work out which URLs affected; if you can do that, then you can really speed up. I mean we also have, we know speed is very important so we’ve got other things, things like technology. So you can actually, but by default ZAP assumes that, well it’s black box testing, it doesn’t know what’s out there. But a lot of the rules are specific to particular operating systems or types of technology, and that is actually the rules understand that. So if you tell us that you’re not using an Oracle database, using MySQL, then ZAP will just use the MySQL rules and it won’t use the Oracle ones that’s actually, I’ve done some tests and that can really speed ZAP up. If you turn off all the technology, it’s actually significantly quicker. Obviously if you’re using that technology you need to turn those things on. But yeah, you can, there’s a lot of ways of speeding ZAP up without actually sacrificing the effectiveness.

Priyanka Raghavan 00:34:41 That’s good to know. And I think that also brings me back to another question that I saw that I was thinking about when I was researching. You just now told us that ZAP can only be used for web application testing, but I do see ZAP running on, I think there’s some tests for running on Raspberry Pi. So is that like the vision that you want to support like say IOT and like smart devices if they support web protocols?

Simon Bennetts 00:35:06 Yeah, we want ZAP to cope with anything which uses web protocols really. So yeah, we’ve got ZAP running on Raspberry Pi; the performance isn’t too bad with the modern ones to be honest. And we know people use ZAP for mobile testing as well. That’s not something I’ve really got involved in, but there’s there’s some articles online we’re trying to link to those. So we want ZAP to be as useful to as many people as possible. IOT isn’t really my thing but we definitely want ZAP to work well in those environments.

Priyanka Raghavan 00:35:35 And finally, before I move on to the next section, since we talked a little bit about the manual and automated scanner, right, which is there, what would be the use case for say an automated versus a manual? Would I start with automated if I didn’t know about the application and then go to a manual mode?

Simon Bennetts 00:35:54 Yeah, I mean the, to do manual testing you have to, it helps to know a bit about security and what you’re supposed to do. If you don’t know what you’re looking for or how to find it, then it’s kind of tricky- particularly cause a lot of web vulnerabilities are kind of weird. They’re not intuitive. So, a lot of the manual features in ZAP we’re kind of thinking it would be web security professionals using them. But you’ve got to be aware that — so, I mean it’s definitely the case that having a pen tester, experienced pen tester, testing your application manually is much more effective than an automated scan. However, it’s also much more expensive. So when I was at Mozilla we would commission a couple of pen tests a year, on different services, and we had a lot of services, so we’d only test a couple of them every year. And we’re talking, I mean there’s like 40 to 80 thousand dollars for one or two weeks’ work.

Simon Bennetts 00:36:48 So, if you’ve got a lot of services, that’s a lot of money, and you can’t do it all the time. But vulnerabilities can be added at any point. So the advantage of a tool like ZAP is you can actually run it overnight; you can run it every day. And it will not pick up all the vulnerabilities, but it’ll pick up some key ones, and if you start getting some vulnerabilities on a service, that could be a good indication that you ought to get some manual pen testers in as well. But you’ll also find that using ZAP means you get more value from your pen tests. When I was at Mozilla I often ran the pen tests or the interaction with the companies doing the pen tests, and it was great to see them come in all confident and after a couple of days they hadn’t found any serious vulnerabilities, and then they started working really hard.

Simon Bennetts 00:37:36 Pen testers are only human, so if they can find easy stuff, they’re not going to put as much effort in. If they can find trivial stuff, which they may well be finding with tools like ZAP, then they’ve got other things to do; they can focus on other stuff. So you won’t get as much value. Whereas, if you find the easy stuff then that’s when you get much more value out of your pen testers. And it’s also great, it’s really useful if you find out early on that, say, a particular developer is interesting vulnerabilities or team, then that’s where you’d start looking at getting more training for these people. It’s finding stuff as early as possible and finding out the causes of how did this happen? Is it a lack of training, do you need different frameworks, do you need… There’s a whole range of things you should be looking at, but finding potential vulnerabilities early as possible is much more cost-effective.

Priyanka Raghavan 00:38:25 True. I think that really rings home very well because I think a lot of the big attacks that we’ve seen in the news is because of the simpler OWASP top 10 vulnerabilities, which cause a lot of like millions of dollars in damages. So yeah, finding the low-hanging fruits maybe are the ones that happen often and then trying to do the pen test in a more targeted way would be a good, that’s good advice.

Simon Bennetts 00:38:50 Definitely. Yeah.

Priyanka Raghavan 00:38:51 The next question I’m asking is a bit interesting in the sense that today it’s the world of AI-powered buddies and AI-powered PR tools. What is the risk of maintaining an open-source tool to scan for security vulnerabilities? So, tomorrow you might have somebody adding some malicious code and then, of course, that gets susceptible to these supply chain attacks, and a lot of the clients get infected because they’ve got that and then you’ve got a AI-powered buddy that’s also reviewing the code or something. So, what do you think about that kind of scenario? How will ZAP cope with that?

Simon Bennetts 00:39:26 AI is a fascinating topic, and I think a lot of people are getting a lot of benefit from using AI –particularly filling out kind of ‘framework’ code. It will speed people up, but I think the people who use it most effectively will be the people who know what they’re doing. And I think there’s a real danger in people who don’t know as much using AI to generate code, particularly if it’s trained on code on the internet because there’s a lot of vulnerabilities out there. And so, I think there’s a very significant chance that AI-generated code will accidentally introduce vulnerabilities; and it’s also possible to poison it, so that it will deliberately introduce vulnerabilities. And if people are using it with less knowledge, there’s less chance of those vulnerabilities being picked up. So, there’s a lot of benefits, but there’s a lot of dangers as well.

Simon Bennetts 00:40:17 And the whole AI thing where there seems to be a big mistake in that we are mixing the control with the data. So, what it means is you can actually tell the AI systems — chat GPT or whatever — what to do, but the data it works on can then change what happens and how the tool works, and that is really dangerous. So, there’s some fundamental problems here, and I’m not saying you shouldn’t use AI systems to help you, but you have to be aware that it’s very risky. And I think we’ll see some significant vulnerabilities introduced in this way.

Priyanka Raghavan 00:40:50 So how will ZAP actually combat those kinds of problems? Suppose somebody in the marketplace produces something that introduces malicious code?

Simon Bennetts 00:41:00 In the ZAP marketplace? So, every change that’s made to ZAP is reviewed by two of the core team. So we will, we’ve got two experienced people who’ll be checking the code, and if it’s even doing something a little bit strange, then that’s when we dig deeper. So, if somebody tried to introduce a malicious code, we would aim to find that as, hopefully, that wouldn’t get through the review process. We do static analysis on ZAP code as well, so we use as many of the security tools we possibly can. But I think in this case the manual review, and it’s not just a case of that we can’t see any obvious vulnerabilities; the code has to be sensible and be doing sensible things. If it’s doing weird things for no readily apparent reason, that would make us suspicious. We want ZAP to be as maintainable as possible and as secure as possible, and we’re aware that people could accidentally introduce vulnerabilities or they could try and deliberately introduce vulnerabilities. So if there’s any code that looks suspicious, that’s when we dig a lot deeper. So yeah, I think that the manual review process for ZAP is the key thing for us.

Priyanka Raghavan 00:42:07 So, the humans will stop the AI generated code, hopefully?

Simon Bennetts 00:42:12 In ZAP? Yes, that’s the idea.

Priyanka Raghavan 00:42:15 Okay, that’s good to know. And it’s interesting, you said that you run a lot of your security tooling on the ZAP code base? Since it’s more a desktop app, how do you do the dynamic testing use ZAP to test ZAP?

Simon Bennetts 00:42:28 We have used ZAP to test ZAP, but yeah, as a desktop tool — or even as a kind of dynamic tool — it’s harder to test, but we do static analysis on pull requests regularly as well. But yeah, the dynamic side we have used ZAP, and we do have a bug bounty so, and we know security researchers have definitely played around with ZAP, so if you can get a remote code execution on ZAP, that’s a thousand dollars, and we’ve paid out three times, I think, for that.

Priyanka Raghavan 00:42:57 Okay. So the next question is what is the process if someone wants to start contributing to ZAP? Can you explain that to our listeners? I’ll obviously add some information on whatever you say to the show notes at the end.

Simon Bennetts 00:43:09 Yeah, so ZAP is a community tool. It always has been. I was originally looking for a community-based tool so I could join. I couldn’t find that and ended up creating that community myself. So in some ways I think it’s easier for a small team to maintain any tool on their own without anyone else getting involved. The ZAP team really believes it’s important for people to be able to get involved. The option for working on a world-class tool like ZAP is really useful and really important, and we’ve had a lot of students working on ZAP. We’ve actually got Student Hall of Fame, A lot of students have worked on ZAP through Google Summer Code and other projects, but you don’t have to be just go via Google Summer Code, anyone can get involved in ZAP. We want people to get involved and we’ll be very happy to help you.

Simon Bennetts 00:43:54 We have a ZAP contributing guide, so just go onto the ZAP website zaproxy.org — and we’ll put links in, I’m sure. But then, there’s a long guide which explains all the things you can do to help us with ZAP, and it’s not just coding. Obviously, coding’s a main thing, but there’s documentation about using ZAP to test things, writing blog posts; there’s a million things you can do. We try and make it as easy as possible for people to get involved. We do know, as a security tool, it’s something developers might be nervous getting involved in. But I mean, I was a developer and I learned security. Quite a few of us have learned security by working on ZAP. Rick, who’s one of the core team, was the security guy and learned how to develop by working on ZAP, has had loads of students who actually made some really key, implemented some key features in ZAP. So we know a lot anyone can make really useful contributions. So you’d like to get in touch having a look at the contributing guide but also just ping me, I’m easy to find online and we’ll include those details. So get in touch and we’ll see what we can do, how we can help you.

Priyanka Raghavan 00:44:55 Okay, that’s great. And finally before I end, there is a question on web application data-leakage problems. I know you said that essentially you can use ZAP to test data problems — or I mean, you can use ZAP with your own data, but suppose I had an example where I have a data-leakage problem and I want to figure out if it’s really an issue. Like, I know that my application has a data-leakage problem. Could I use ZAP for that? Would I, just as a novice person, like should I be looking at some APIs that with particular calls being made to the database, find that and then try to use ZAP for that? Or how could I find out if my app has a data-leakage problem?

Simon Bennetts 00:45:34 It’s kind of usually a blended approach is best. Now one thing with ZAP, we’ve got some passive scan rules which will attempt to detect data leaks. They’re typically reported as either informational or low, but it’s still, but things like credit card numbers, we spot numbers that look like valid credit cards and user information. So, we will report information like that. So, one of the key things for any tool like ZAP is how effectively you explore your application. Now the most effective way for an application designed for humans is to get the human to do that. So, you can actually start ZAP, you can launch browsers from ZAP, and then you can explore your application manually. And if ZAP spots any data that we know to be interesting being leaked, then that will be reported to you. And if there are particular things you’re looking for that we’re not looking for, then you can create your own scan rules.

Simon Bennetts 00:46:31 So you can create your own passive scan rules. You don’t have to be programmer to do that. I mean, obviously some programming knowledge helps, but you can write those things in scripts and we’ve got a load of example scripts in the community scripts repo. So we’ve got examples of how to do these things. So you can actually write some custom rules in a scripting language of your choice — we support quite a few — which looks for things that are very specific to your industry, to your application, to your company. And then, as long as you explore the application effectively, ZAP will report those things. If you don’t have time to explore it manually, we can explore it with, we’ve got two spiders — one a traditional spider which is very fast but can’t handle modern applications as well. Then we’ve got an ajax spider which launches browsers to handle the ajax side for the modern applications and clicks on things. We can also import API definitions. So whether it’s SOAP, open API, GraphQL, all these kind of things we can import. So if you can explore your application effectively, then ZAP will tell you what data gets leaked.

Priyanka Raghavan 00:47:33 So what I’m hearing is if you tweak ZAP in the right way, then you’ll be able to find out if you have a data leakage problem or not, yeah?

Simon Bennetts 00:47:41 Well, I mean we’ll look for some standard things, but if it’s not standard then yeah it’s very easy to extend ZAP to look for whatever you want.

Priyanka Raghavan 00:47:49 Okay, that’s great. And I’ve actually forgotten to ask this question, but one of the things that you said is of course you don’t have to only check the UI part of it, you can also check APIs as well, right? So having that option, that’s pretty powerful.

Simon Bennetts 00:48:03 And then if you’re using some weird format we don’t recognize, then you can still just — if you can proxy another tool through ZAP, then you can get that to invoke your API or do whatever you need to do. So, we try to support all the standard API definitions. If there’s something we don’t support and you think we should then let me know, but you can just proxy any other tool through ZAP as well.

Priyanka Raghavan 00:48:27 I think it’s quite a comprehensive list of questions that I’ve asked you and we’ll find out later how it goes. Finally, how does one find you? Are you, would we have to go on the website or are you, would I want to say one of these social networking websites like Twitter? Are you like active there, or…?

Simon Bennetts 00:48:44 Yes, very much so. So on the ZAP website we have a community, then a team link, and the ZAP core team are there and links to all of our social networks. I use the username psinon. So, that’s what I’m on Twitter, on GitHub, all these other things. So you should be able to find me. If you can’t find me then you’re really not trying very hard , but we’ll include some of the key links.

Priyanka Raghavan 00:49:11 Yeah, I’ll definitely add a link to your Twitter handle and of course GitHub as well. So it’s been great having you on the show Simon, thanks for coming. Is there anything else that you want to tell us before we sign off?

Simon Bennetts 00:49:24 Just thank you again for having me. It’s been a pleasure talking to you, and we do want people to get involved, so if you want to get involved, please do just get in touch. And we are looking for companies to support ZAP in the same way that Jit does. So if you’re using ZAP — a company using ZAP — and you’re interested in helping us out, making ZAP even better, then please get in touch with me as well.

Priyanka Raghavan 00:49:45 Thank you. This is Priyanka Raghavan for Software Engineering Radio. Thanks for listening.

[End of Audio]

Tagged as:
Join the discussion

More from this show

Menu

Recent posts