Stanisław Barzowski of XTX Markets and a committer on the jsonnet project joins SE Radio’s Robert Blumen for a conversation about the jsonnet programming language. A superset of JSON, jsonnet adds programming language capabilities, particularly to address the need to handle large but mostly repetitive JSON configurations. They discuss the project’s history, use cases for Grafana and Kubernetes config, and interoperability with YAML. They examine jsonnet details, including the command line, constrained capabilities of the language, and objects and inheritance, and then consider the toolchain: compiler, formatter, and linter, as well as test frameworks and testing, package management, and the language’s performance. Barzowski describes four implementations — go, C++, Rust, and Scala — as well as popular libraries and the standard library.
Show Notes
- Jsonnet – Jsonnet Configuration Language
- Jsonnet – Language Reference
- Jsonnet – Standard Library
- Jsonnet – Kubernetes
- k8s jsonnet library
- Grafonnet
- Grafana as code
- Automating Grafana Dashboards with Jsonnet
- Declarative Infrastructure with the Jsonnet Templating Language
Transcript
Transcript brought to you by IEEE Software magazine.This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Robert Blumen 00:00:16 For Software Engineering Radio, this is Robert Blumen. Today I have with me Stanislaw Barzowski. Stan is a graduate of the University of Warsaw. He’s a Developer at XTX Markets in London and formerly was a software engineer at Google. He is a committer to the Jsonnet open-source project, and that is what we will be talking about today. Stan, welcome to Software Engineering Radio.
Stanislaw Barzowski 00:00:42 Hello, Robert.
Robert Blumen 00:00:44 I’m very interested to talk about Jsonnet with you. Jsonnet is closely related to JSON. I’m guessing most of our listeners have been exposed of that, but could you give us a quick summary of JSON and then we’ll move on to Jsonnet?
Stanislaw Barzowski 00:01:00 So JSON is a very simple, I would say mostly a data interchange format — just a tiny subset of Java script. But actually like the Javascript part isn’t that important. It’s only the historical part. It’s just how everyone saves data these days. So yeah, and Jsonnet is much, much more. Jsonnet is a programming language. It’s a language for building configuration. So you write your program in Jsonnet to generate your JSON, but it happens that if you have valid JSON, it’s also programming Jsonnet, which generates with JSON, which is quite convenient, hence the name.
Robert Blumen 00:01:38 What does Jsonnet add that is not there in the original JSON?
Stanislaw Barzowski 00:01:43 So it adds a natural programming language features that you might want in your configuration file. So starting with variables, then functions for templating first, a form of object orientation, which is somewhat non-standard — I believe we’ll talk about this in a bit more detail later — and ability to input other files so you don’t have to have your big fig all together. And you can also reuse pieces in different configuration files.
Robert Blumen 00:02:12 I’m going to put two questions together and you can answer them at the same time or one after the other. But why was Jsonnet created, and what do you get from having these additional features that are not there in JSON?
Stanislaw Barzowski 00:02:28 Okay, so the original motivation was to create an open-source solution to something similar to what was used internally at Google. So you have this need when you have a lot of microservices or, in fact, any sort of situation. You have like hundreds of configuration files of, we’re using some logic, some parts between them so we don’t have to repeat the same file name over and over again, for example. Or to make sure things are consistent between your configuration files. And Google has an internal language for this purpose, but around the time when Kubernetes became popular, they’ve had an idea to create an open-source language sort of like that with some lessons learned. So I would say better structured, in some ways, though actually quite a bit different in implementation and yeah, so in 2014, JSON had started as a 20% project at Google.
Robert Blumen 00:03:24 So, I often find with JSON I have some very long files, and one of the things you mentioned is you have a lot of files which are quite similar but only slightly different, or within the same file I have big chunks of code that are very much repetitive but they’re not exactly the same. And if I could just say I want this block here that’s 90% the same and I want to just change these three things, there isn’t really a natural way to do that with JSON, but a programming language should enable me to do things like that where I could have a chunk of code and then I reuse that code. Is that some of the motivation for this project?
Stanislaw Barzowski 00:04:04 Yes, exactly that. So you sort of have the cycle when like you introduce some configuration to your program. At first, you start with configuration inside your program just like as part of your source. Then you start extracting it. You first have like a simple configuration file with just a few parameters, but then as it grows it turns out that you need variables, you need to reuse parts of this. And then like you want to have some templates because things are not always exactly the same. Then you want to add inputs to share things between files, and then you sort of reinvented the whole programming language just specifically for the purpose of your program — usually in a quite ad hoc basis so the outcome of this process usually isn’t of very high quality. So what Jsonnet does is sort of gives you a generic solution to that problem.
Stanislaw Barzowski 00:04:54 It gives you like something that you can use to have this complex configuration and manage complex configuration for your software. And the two ways to approach that: one is from the offer of a program perspective, which is oh, I don’t want to invent my new programming language just to specify my complex config in a convenient way. So I can just use JSON and suggest that someone can use Jsonnet or one of the comparable solutions instead. And actually, I don’t even need to care which one the user actually chooses in the end. And the second perspective is sort of from the user’s point of view. So let’s say I have a lot of microservices, or just a lot of configuration modernization, and I can have a centralized mechanism to generate all of this configuration and share some parts between them.
Robert Blumen 00:05:50 Just now you said something I hadn’t thought of. The way I would have seen this being used, and the way I’m using it in my job right now, is we have a tool chain that runs into Jsonnet programs and generates JSON and pushes those out to where the configuration can be consumed. You mentioned that you could teach your program to use Jsonnet as the configuration language. So, the program instead of reading JSON would incorporate Jsonnet. In that situation, would you embed the Jsonnet language into your program and have it process your config files?
Stanislaw Barzowski 00:06:26 You can’t do that as sort of a quality of life feature. But what I would normally recommend is just to consume JSON and then like you don’t even need to impose Jsonnet specifically on your users. When the user can make a final choice whether to use Jsonnet or DLL or whatever they like, or even if their use case is simple enough, just JSON. They then they don’t need all of this extra complexity. So, you can just not care about this at all. Just consume JSON or YAML, and the users can generate their JSON that you can consume with Jsonnet.
Robert Blumen 00:07:02 My understanding is YAML is a superset of JSON. Does Jsonnet interoperate with YAML equally well?
Stanislaw Barzowski 00:07:10 In a way, yes, because Jsonnet generates JSON, and JSON is a subset of YAML. If you generate JSON with Jsonnet, this is automatically also YAML. So in this sense, you can always — it’s not the JSON that is generated by Jsonnet, it’s not idiomatic YAML but this can be automatically transformed if you really care. But yeah, you can just take the JSON that is emitted by Jsonnet and have it interpreted as YAML, and it will work.
Robert Blumen 00:07:41 We’ve been using at my job another library that’s written in Jsonnet, called Grafonnet, which generates JSON input for the Grafana Dashboard system. One of the things I found very helpful about this is Grafana has an enormous number of parameters of which many of them are not well documented, and I don’t know what they all mean. The library provides rational defaults for almost everything. So, I can get productive on a programming task without understanding every single variable. Whereas if I was writing it directly in JSON, I would have to know quite a lot more. So by abstracting and defaulting it makes me much more productive. I want to ask, are there other libraries or packages in the ecosystem similar to that, that you’re aware of for other services that use a lot of JSON?
Stanislaw Barzowski 00:08:37 There are a few. I’d say that probably the most important ones are for Kubernetes. A few of them, I think the best one currently is just called Kubernetes under JSON Eclipse organizational GitHub, but there are a few out there. But like this is mostly useful in cases of like actually very complex config formats. But I think you can very successfully just like use your vanilla Jsonnet to generate your config. This is very much like an optional thing to have a library specific for your output configuration.
Robert Blumen 00:09:10 In the Kubernetes case then, does Jsonnet and the associated libraries become a competitor to tools like Helm?
Stanislaw Barzowski 00:09:18 I’m not sure. Maybe on some level. I don’t have a very strong opinion here. I think sometimes they’re used together to some extent. I’m not sure.
Robert Blumen 00:09:28 Could I, if for some reason I didn’t like Jsonnet or just trying to make a point about something comparable to this, could I do all my programming in my favorite language, whatever that is? Let’s say Python, which has data structures and has all the programming language features, and then at the end export to JSON? What would be the pros and cons of that versus Jsonnet?
Stanislaw Barzowski 00:09:52 Yeah, I think this is a very valid approach. Why would you want to use Jsonnet instead of like whatever you are currently using? So far, a few things. So first, if your language is something like C++, then there’s a lot of boilerplate just like managing these extra files. Things like the way you express JSON is just not very ergonomic. So it’s much easier to, especially in the fairly lightweight configuration where you use it most, there’s just like, here is my JSON output with just these few things being changed dynamically. In such cases, yeah, just Jsonnet offers you much better like day-to-day developer experience. And second thing is hermeticity, and this is sort of a funny feature because this is about what we don’t do rather than what we do. It is a guarantee that Jsonnet won’t randomly read some files or have any additional side effects. So, in principle you should be able to take your whole Jsonnet code base, run it five years from now in a completely different architecture, and get byte-for-byte the same result. And found no hidden dependencies, Jsonnet programs cannot just like the, all of the files that Jsonnet can access is known statically.
Robert Blumen 00:11:10 I’ve heard this word before; I had to look it up the word ‘hermetic’ in dictionary.com, a hermetic seal ensures perfect waterproofing.
Stanislaw Barzowski 00:11:19 Yeah, this is what we are getting at. So the idea is you can think of Jsonnet program as a mathematical function, not only in the implementation detail sense, but there are actually no side effects, no unexpected interaction with the file system or the network, or anything like that. Which in a general purpose language, that would be a fatal flaw. But for configuration language you don’t really want all of these things to be happening. So you have this separation, you have well-specified inputs which produces your well-specified output.
Robert Blumen 00:11:52 So where I think you’re going with that is if you run a Python program, it could pretty much do anything you want, and in security in the supply chain, we really want things that cannot do anything they want because they could do a lot of harm.
Stanislaw Barzowski 00:12:05 I wouldn’t stress the security aspect here. I would say it’s mostly about making things understandable because normally config files are not super well-tested, and that’s like there’s really no need for all of this extra interactions. Like usually it means that someone is taking some weird shortcuts, but actually like you want what determines your configuration to be very well understandable, and also you might want to run it in your CI environment or something like that. And then all of the extra dependencies on the environmental it runs in are potential headaches. So this is the primary motivation. In terms of security, well I would say that yes, it’s safer to run arbitrary JSON than run arbitrary Python, for example. This is the programming language, and this is still like arbitrary code execution, which may take arbitrary amounts of time. So treat Jsonnet code as code, not as data.
Robert Blumen 00:13:02 I want to move on more now into the language itself. What does a Jsonnet program look like?
Stanislaw Barzowski 00:13:09 So the simplest Jsonnet programs look like just JSON, but then you can add a few additional things. So the most important one I’d say is functions and function calls. So you can just define your own function, which effectively parameterize part of your JSON, use it in multiple places. Variables are even more basic when you are just specifying one specific thing.
Robert Blumen 00:13:36 How do you run a Jsonnet program? What happens when you run it?
Stanislaw Barzowski 00:13:40 So, normally Jsonnet programs are run in one of two modes. One is just like you run the Jsonnet command and you specify all of the extra inputs that are injected. As I said before, it cannot just reach out for things. So you need to pass everything explicitly, and then it is going to evaluate the expression that your program is and output your JSON — or in fact, there’s also a mode which lets you output arbitrary strings. So for generating any other config files.
Robert Blumen 00:14:11 So, unlike a general-purpose programming language, you don’t have to say print F or any type of output command. The purpose of the program is to run to produce its output, which is JSON, and that’s what it does every single time, correct?
Stanislaw Barzowski 00:14:26 Exactly. It doesn’t necessarily have to be JSON in the output; it can also be any other data, but it is JSON by default. So if you just have a program which consists on a single string, the output will be a JSON string. But if you run it with dash S, then it will actually output that string as just that — not as JSON without the quotes. So this is how it normally works. There is also a library API, so you can also call it directly from your programs and then yeah, you don’t use command line online interface.
Robert Blumen 00:15:01 In that case, it has an API, then what different programming languages support that API?
Stanislaw Barzowski 00:15:07 So there are a few. There is CAPI, which is I would say the most generic, and you can use these bindings from whatever you want. There are also C++ bindings, there is the Go API, there is an alternative implementation. Rust, I’m not sure if it has Rust API? But I would guess it has? There is also a Scala implementation with API for JVM.
Robert Blumen 00:15:31 Like a lot of programming languages these days, it incorporates ideas from all over the programming language map. So, we could discuss some of these different concepts. One is that it is largely declarative. Could you delve into that a bit?
Stanislaw Barzowski 00:15:48 I’m not sure what you’re getting at with declarative, but what I can say is like it’s pure, and the order of evaluation doesn’t really matter. So, you pretty much think of a program mostly as a function, like a mathematical function, and you think of everything in terms of its definition. So, in that sense it’s declarative. So, like you’re not thinking in terms of hey, here is a program which tells the machine to do this and then that and then something else; you are saying, oh here is a value which consists of these parts, and then it’s up to Jsonnet interpreter to evaluate them in a reasonable way.
Robert Blumen 00:16:31 Where I was going with that is: JSON document is a Jsonnet program, and if I ran it I would get a document back; maybe I wanted to add a single variable that would be calculated by a function and make that be a value of a field in the document. So, I can add that and I run the program, and now I get the document but it’s interpolated the value of the variable. The program is essentially a document, and I can manipulate different parts of the document through these programming language constructs, but it’s really just a document. That’s how I saw it. Am I thinking about it the wrong way?
Stanislaw Barzowski 00:17:11 Yeah, I think this is exactly correct. So yeah, like I think the best practice is to have a Jsonnet file for each output file that you want to produce and think in terms of what parts does it need to have, and then have appropriate object fields or whatever you need inside that file, and then it can import some additional files for anything you want to have reused.
Robert Blumen 00:17:37 Jsonnet also incorporates some object-oriented concepts. Can you explain the object-orientation model?
Stanislaw Barzowski 00:17:44 Right, so we start with just like normal JSON objects, which are just key-value maps where keys are strings and values can be any Jsonnet values which are JSON values and functions. Functions are one additional type that we have on top of what JSON has. But we also have a quite unique concept of objects overriding each other, which is how object orientation works in Jsonnet. And the way to think about it is an object is a stack of layers. Each layer has some fields, and when I write A plus B, when A and B are objects, then the new object — the resultant object — is first all of the layers from A and then all of the layers from B on top. And then when I use any self-referential constructs, which are self or super, it looks through they layers to find the appropriate field to access.
Stanislaw Barzowski 00:18:45 So if somewhere in my one field of my object uses self.foo, what happens is it will start looking from the top of a stack of layers for a field called foo, and super similar to how super works in object-oriented languages. And in the stack of layers terms it starts looking for your super.foo in the layers below the current layer from which it is called. So it is a mix-in-based inheritance model where you can treat every single object as a mix-in. So yeah, there are no declarations in Jsonnet whatsoever. Everything is just an expression. So this is how you can get inheritance without any declarations.
Robert Blumen 00:19:34 Use the word mix-in. And I’m aware that many languages have a concept of mix-in. Can you explain a bit more what do you mean by mix-in within Jsonnet?
Stanislaw Barzowski 00:19:46 Yeah, so mix-in is just an object which expects to be used to overwrite another object. So for example, it might add an additional field. I know, let’s say I have a web server configuration, and I can have a mix-in which adds HTTPS configuration to my route or something, and it can refer to the fields from the whole object.
Robert Blumen 00:20:12 So in order to add HTTPS, then I might need to add some additional keys and values that are not in the base object, or maybe I might have to change the way something is done in the base object. And it’s possible to both add and change things with mix in, is that right?
Stanislaw Barzowski 00:20:30 Exactly. You can either add new fields or override existing ones. It is not currently possible to remove false completely. You can override them to null, which usually is good enough.
Robert Blumen 00:20:41 So I like that web server example. Could you give another example of how inheritance was used that you’re aware of to do something useful? Not necessarily involving a mix-in, but an example that shows the object-oriented feature of Jsonnet?
Stanislaw Barzowski 00:20:57 So I think all of these examples are quite similar and these are for the use of actual fully fledged inheritance, and that is you have different aspects of your configuration. So it can also be, I don’t know, your rate limit or some resource limits — depends on what you’re trying to configure, obviously. And one style of writing Jsonnet is to combine a bunch of mix-ins. Each of these mixins is responsible for one aspect of configuration, and then you can just use the operator plus to combine this into your final full configuration.
Robert Blumen 00:21:34 And depending on your use case, you might find the same mix-in could be used across different root documents. Is that right?
Stanislaw Barzowski 00:21:41 Yes, yes. And for like some of this libraries also just like offer mix-in which can be used by anyone. So for example, for configuring Kubernetes, it’ll automatically create the right fields, et cetera.
Robert Blumen 00:21:55 Right because many Kubernetes objects, although the type is different, they contain a lot of the same elements in the body. So there’s definitely opportunity for some reuse there.
Stanislaw Barzowski 00:22:07 That’s right, that’s right.
Robert Blumen 00:22:08 And if you have a strict hierarchical object orientation, it’s difficult to reuse things that are in a derived class because you can only derive from one base class, whereas the mix-in enables you to take the same derived behavior and apply it very flexibly.
Stanislaw Barzowski 00:22:28 Yes, that’s true. And Jsonnet offers one more thing on top, which is quite unusual: You can create an Uber mix-in out of multiple mix-ins. So the plus operator is used for inheritance in Jsonnet, which is quite an unusual choice, but it actually is related to the semantics because inheritance in Jsonnet has a lot of similarities with concatenation because it’s just concatenating with layers. So, it’s an associative operation and you can build all the mix-ins you need before you add it to the actual thing. You don’t need to start with a base layer and then add it one by one. You can combine them together first.
Robert Blumen 00:23:08 In some of the documentation, Jsonnet is described as a functional programming language, what are some of the functional programming features of Jsonnet?
StanislawBarzowski 00:23:18 Yes, I think it’s more functional than object-oriented, especially when used well. So the primary functional characteristic is purity. So, you structure your programs by writing expressions to evaluate to a single value; there is no mutable state whatsoever. And I would say that the default way of how to parameterize things is by introducing functions. So, you can in fact wrap any Jsonnet expression in the function and parameterize it this way. And because Jsonnet programs are composed entirely of expressions, this is actually very powerful. You can take your entire file, wrap it in a function, and now you have a parameterizable module.
Robert Blumen 00:24:03 Does it accept functions as arguments to other functions?
Stanislaw Barzowski 00:24:07 Yes, yes. Functions are regular values. You can pass them around freely; they are completely first-class objects. The only thing you cannot do with functions is you cannot print them out or compare them. So, but this is quite expected, I believe.
Robert Blumen 00:24:25 In the documentation, there’s a section about the lazy semantics — that you can modify something while you’re creating it. Can you explain how that works?
Stanislaw Barzowski 00:24:36 Yes. So, Jsonnet is a lazy language, and that means that every expression is evaluated only at the point when it is needed. This is similar to how Hascell works, for example, but it’s not actually like anything esoteric. The main benefit of that is you don’t need to worry at all about order of how you define things. You can have one object field which refers to another object field. So, let’s say you start with a simple JSON object with no extra logic whatsoever, and you start parameterizing it, and you decide that one field should depend on the value of another field; you can just do that and you don’t have to worry about, oh but which field is actually created first? As long as there are no actual cycles in the definitions, it will all just automatically work for you.
Robert Blumen 00:25:29 And how is that helpful, or can you give an example of what you might do with that?
Stanislaw Barzowski 00:25:33 I’d say it’s mostly just a quality of life thing because technically you could always just extract things in the right order, but you don’t have to think about this at all. You can extract every definition you have in your code to a variable, and you don’t have to worry about, ‘oh but actually this needs something which is defined later.’ And this is especially important when the data is deeper. So, you have objects which have objects as its fields, which in turn have objects as fields. So you can have object A, which depends on some fields of object B and then object B in terms depends on some other fields of object A, but as long as there are no actual cycles on the level of individual primitive values, then it’s all fine.
Robert Blumen 00:26:25 Now, given the nature of what you’re trying to do and how constrained it is, I’m going to guess there’s not really a need for things like exception handlers. It either the program runs or it doesn’t, and if it runs you get an output. Is that correct?
Stanislaw Barzowski 00:26:41 Yes. We have a primitive form of error handling, which is just, you have an error expression which just throws a fatal error. There’s no error recovery. Yeah, there’s really no need because the idea is Jsonnet never needs to interact with any external systems, so there’s no need. This is just all pure computation. Why would you want to, right?
Robert Blumen 00:27:06 Right. So it’s not sitting there retrying to connect to the database until it succeeds.
Stanislaw Barzowski 00:27:12 Exactly. So there is no need for any advanced error handling, and in general, we are trying to keep the language minimal and very, very simple. So if something isn’t needed, it’s not there.
Robert Blumen 00:27:26 Now, covering the language features, I want to go into the tool chain. You mentioned before that there is a Jsonnet command that you can run it. Are there other auxiliary commands — like, is there a formatter, package manager, any other parts of the ecosystem?
Stanislaw Barzowski 00:27:44 Yes, so the formatter is a very important one. So, it’s normally distributed together with the Jsonnet interpreter; it’s called FMT, and it has a few configuration options, but it’s also a bit opinionated. And there is also a linter, which actually was a bit of an engineering challenge because we have no types whatsoever, and yet we have fairly complex structures to work with, and field names of objects can be calculated dynamically but usually aren’t. And to deal with all of that, a special linter was created which sort of constructs a set of equations on sets for each value in the program and then tries to solve which values can appear in different places of the program. And mostly, it catches things like using a field which doesn’t exist.
Robert Blumen 00:28:38 Can you think of any other types of errors that the linter could find?
Stanislaw Barzowski 00:28:42 It can also find some classes of infinite loops, and things like unknown variable names are caught by the basic interpreter. So, in fact, all of the usages of variables are always statically known.
Robert Blumen 00:28:56 We talked a little bit about there is some ecosystem of libraries. Does Jsonnet offer a package manager?
Stanislaw Barzowski 00:29:04 So there is a third-party one; it’s called Jsonnet bundler. There is a bunch of packages available. It’s not super mature yet, but it works. And to the extent that you actually need libraries, I think it’s pretty good.
Robert Blumen 00:29:19 Is there any type of testing framework, or whether you say yes or no, how do people go about testing Jsonnet programs?
Stanislaw Barzowski 00:29:28 So, there exists a testing library, but like I would say the number one way to test Jsonnet programs is for technical golden testing. So you would just save the value of your output, of known good output, to a file and then whenever it changes you would just inspect the diff in the output. This is how we test Jsonnet interpreter itself. We just have a bunch of examples of Jsonnet programs, and we compare the output against the known good one. But in some cases, it’s easier to have something more akin to unit tests; there’s a built-in support for assertions in Jsonnet. There are also a few third-party libraries.
Robert Blumen 00:30:14 And moving on, what language is it implemented in? How many implementations are there, and what languages are they written in?
Stanislaw Barzowski 00:30:24 So, there are four implementations which I would call ‘production-quality’ — the original one, written C++, which isn’t very efficient. Then there is a GO implementation, which is the closest one to the official one currently. This is the one that I spend most of my time on. There is also an implementation in Scala, created by Databricks. I haven’t used it but it’s known to be quite fast. There also a new implementation in Rust called JR sonnet. And if you’re wondering why there are so many, it’s because Jsonnet is a language with first a very simple language, then it has a well-defined specification, there are actually formal operational semantics for the language on the website. So you should be able to know exactly what every construct does. So, it’s actually like, I think, it’s possible for a decent engineer to write acceptable implementation of Jsonnet within months.
Robert Blumen 00:31:26 In my use of it, I’ve been generating files that are a few thousand lines of JSON, and I’m not sure which implementation I’m using, it’s whatever I got on the Mac with BREW; it runs quick enough that I don’t notice it to where I wouldn’t care if it was any faster. Has there been any benchmarking, or maybe you could just speak from experience of how long does it take to run for different size of either input code base or outputs?
Stanislaw Barzowski 00:31:55 It’s hard to say generally, other than like it is still pretty slow — even the fast implementations. So performance was never like a major goal because it’s just for generating configuration. So as you said, in many cases it’s not noticeable right away. And I’m not sure about home BREW, but it might be using actually the original slowest implementation of them all. But, once your output is big enough, once you have, for example, a thousand Jsonnet files, then like it can become a bit annoying. So actually the motivation for creating the Scala implementation, which was chronologically the second one — came before the Go one — was just to make it faster.
Robert Blumen 00:32:41 These different implementations, are they all at the same stage of maturity, or are you still seeing much traffic in terms of issue reports on any of them?
Stanislaw Barzowski 00:32:52 So, I have most familiarity with C++ and Go ones, which are the sort of ‘official’ ones. And by official I mostly mean that they are maintained by Dave who is the original author of the language. And like, we definitely still see issues. They are about smaller and smaller things, like additions to the standard library or actually creating these external tools. So Jsonnet linter was a 2019 early 2020 edition, and we still see a steady stream of PRs, but the language itself doesn’t change very much. This is mostly about, additional tooling extension, the standard library, et cetera.
Robert Blumen 00:33:35 Standard library, now I realize I wanted to ask about that, so I’m glad you mentioned that. What are some of the offerings of the standard library?
Stanislaw Barzowski 00:33:42 These are mostly very obvious things that you would expect in the language. So, min and max functions, some basic string manipulation, you know, splitting strings by a dot if you need to, converting ASCI to uppercase. Parsing various values you can parse JSON, parse YAML, there are also some functions which let you automatically produce various formats. So you can produce an ini file or a Python object or, in fact, a JSON but within your program. So in some cases some weird formats require some JSON embedded as a string. So yes, you can do that. Ugly as it is. Or XML or TOMO, I think these are the ones that we currently include. Yeah, some array functions, like basically what you’d naturally expect, but without anything that relates to external interactions because that’s not allowed in Jsonnet. So, like no files, no network.
Robert Blumen 00:34:42 You haven’t mentioned some of the functional programming primitives are in the standard library, like map and fold.
Stanislaw Barzowski 00:34:49 Right, right. So these have usual definitions. So, you can write JSON’s programs in the standard like, functional way with all of the usual goodies.
Robert Blumen 00:34:59 Is the standard library implemented in, is any of it implemented in Jsonnet or is it all implemented in the language of the implementation?
Stanislaw Barzowski 00:35:09 So historically, almost all of the standard library was implemented in Jsonnet. This turned out to be quite slow. So the current approach is every new addition to the standard library needs to have a Jsonnet implementation to guarantee that there is some reasonably simple definition, and it can run on every platform, et cetera, et cetera. But implementations also provide built-in versions of these functions, and we test their output much as of a reference Jsonnet implementation. But this is only a performance optimization because Jsonnet as a language is fairly slow. So by implementing some built-in functions in, let’s say, Go, we can get often like order of magnitude improvements.
Robert Blumen 00:35:55 Is there an extension point if programmer wanted to add a function in C or Go to their own library?
Stanislaw Barzowski 00:36:04 Yes. For exists a native function API, you need to use Jsonnet as a library to have that available. It’s not very encouraged because it effectively creates a tiny dialect. But whenever, let’s say I’ve used some complex logic from your project, then you can call that from Jsonnet. The native functions are still supposed to be pure. So things like adding IO operations, adding file operations for native functions is very discouraged. Technically some people do that with works.
Robert Blumen 00:36:39 We’ve covered pretty much what I wanted to about the language itself. If listeners would like to get started, where can they go?
Stanislaw Barzowski 00:36:46 So to start using the language, the best place is Jsonnet.org. You can find pretty much everything I said, only much more clear. And we have a bit of an interactive playground. So you can see how exactly it works on some examples you can play with them, change something, see how it affects the output. There’s a tutorial and a very detailed language reference. And for any potential contributors, I recommend just going to the GitHub page, open an issue, open a PR. We try to respond in a reasonable timeframe. And I think Jsonnet is a quite nice place to start contributing to open-source just because it’s quite self-contained and quite well-defined. So it’s just quite easy to get started.
Robert Blumen 00:37:35 And if listeners would like to find you anywhere on the internet, where would you direct them?
Stanislaw Barzowski 00:37:40 So, if you want to ask me a question about Jsonnet, the quickest way is through Slack. So there is a Slack channel on Kubernetes Slack called Jsonnet, #Jsonnet. This is probably, if you mention me, that’s probably the quickest way to get hold of me. Otherwise, yeah, for Jsonnet things open an issue on GitHub, or you can always message me on LinkedIn for other matters.
Robert Blumen 00:38:05 Is there anything else you’d like listeners to know about Jsonnet before we wrap up?
Stanislaw Barzowski 00:38:10 I think we’re good. I think we’re good here.
Robert Blumen 00:38:12 Okay. Well Stan, thank you very much for speaking to Software Engineering Radio. It’s been great.
Stanislaw Barzowski 00:38:18 Thank you very much.
Robert Blumen 00:38:19 For Software Engineering Radio. This has been Robert Blumen. Thank you for listening. [End of Audio]
