Jérôme Laban, CTO of Uno Platform, joined SE Radio’s Jeff Doolittle for a conversation about cross-platform user interfaces. They discussed the unique challenges and possibilities related to applications designed to run on multiple platforms — issues such as platform fidelity versus product-specific fidelity, including the trade-offs between a lowest-common-denominator approach and embracing each specific platform’s unique interaction patterns and presentation styles. They also explored the risks of over-reliance on tools over design goals, as well as the unique challenges related to prioritizing technical issues in relation to business goals and customer expectations.
- Cross-Platform UI Vendors and Frameworks
- XAML overview – Visual Studio (Windows)
- Custom Software & Mobile App Development – Cloud Migration Services | Inventive
- Log4j – Apache Log4j 2
- Figma: the collaborative interface design tool.
From the Computer Society
- A Performance Evaluation of Cross-Platform Mobile Application Development Approaches
- Automated Cross-Platform GUI Code Generation for Mobile Apps
- Mobile Terminal Development Plan of Cross-Platform Mobile Application Service Platform Based on Ionic and Cordova
- Cross-Platform Development Tools for Smartphone Applications
- Cross-Platform Mobile Phone Game Development Environment
- Cross-Platform Instant Messaging System
From SE Radio
- Episode 266: Charles Nutter on the JVM as a Language Platform
- Episode 300: Jonathan Stark on Mobile App Development
- Episode 427: Sven Schleier and Jeroen Willemsen on Mobile Application Security
- Episode 428: Matt Lacey on Mobile App Usability
- Episode 437: Tim Sneath on the Architecture of Flutter
- Episode 452: Scott Hanselman on .NET
Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number and URL.
Jeff Doolittle 00:00:18 Welcome to Software Engineering Radio. I’m your host, Jeff Dolittle. I’m excited to invite Jérôme Laban as our guest on the show today for a conversation about cross-platform UI. Jérôme has been programming since 1998 and is a Microsoft MVP. He has been involved mainly in .NET and C# development as a teacher, trainer, and consultant in France. He is the CTO of UNO Platform in Montreal, the first and only C# XAML platform for single code-based applications that run on Windows, web assembly, iOS, Mac OS, Android, and Linux, which I think covers pretty much everything. So welcome to the show Jérôme.
Jérôme Laban 00:01:01 Hi, Jeff. Thanks for inviting me.
Jeff Doolittle 00:01:02 Glad you could be here. So, today we’re going to be talking about cross-platform UI. So, let’s start. What’s the problem space, and why is there a need for user interfaces that can run on multiple platforms?
Jérôme Laban 00:01:15 The industry has been trying to chase that tail for quite a while, and the big problem is all about skillset and duplication of issues and being able to reproduce anything, or having different teams, different silos, and things like that. So, trying to get teams to be able to reuse as much as they can across platforms without having to be impacted by the underlying platform as much as possible, and depending on what you’re targeting, that can be very difficult. In some cases, if you’re trying to get to let’s say iOS and Android, those two can be very different when you’re trying to get to the UI. And to most extents, same thing goes through a non-UI API. So, if you want to go to geolocation or things like that, and it’s very difficult to get to something that works across platforms. There’s a web that tries to do this with getting across platform.
Jeff Doolittle 00:03:15 Yeah. And there’s a lot you covered there as far as developer skillsets, for example: which platforms they need to know. And I imagine in a large company that can afford the funding, they could have different teams that know different platforms and languages more than others, but of course, then there’s still coordination issues if you want consistency. And so either way, it’s all going to be trade-offs at the end of the day, between developer productivity, fidelity for what you want your product to look like, and how users are going to interact with it. And so, obviously those are just some of the many challenges, maybe in a bit more of the details. You’ve been doing this for a long time, so I know there’s a lot of design considerations that can make cross-platform applications a challenge. So, maybe what are some of those specific things that teams might face just from a design standpoint when they’re trying to adopt an application delivery model that lets them, as you said, reuse code and reuse applications that are going to work on different platforms from the same code base?
Jérôme Laban 00:04:15 Two sides of that coin. So, if you want to reuse the code, then that means that you’re going to have to make sure that the platforms you’re trying to use is going to cover as much as you can, as much as possible, from what the underlying platforms are going to be providing. So there are two ways for doing this. The first one is lowest common denominator. So that means that you’re going to get something that looks the same, behaves the same at the expense of what the underlying platform is able to provide. And that’s a big problem because if you’re on iOS, then you may expect, let’s say a text box or an input control to behave in a specific way because the platform is behaving something in a specific way, and even more so in some cases you want to do, let’s say accessibility, for instance. Accessibility is one big problem.
Jérôme Laban 00:05:02 And if you’re ignoring what the platform provides, then you’re going to get all sorts of issues with regards to APIs may not be there for doing what the platform provides. So let’s say you want to do screen reading, then you may be having some issues there. So if you’re trying to get to something that’s lowest common denominator, then you’re going to get into trouble for those kind of things. Then there’s the other way around. If you’re trying to get access to everything that the platform provides, then you can get into some of the weird parts where you can’t have the branding that you want. And then let’s say the button on iOS doesn’t allow you to have something that looks like the button from your company and that your designers want. So that’s the kind of things that makes it a bit difficult in terms of UX-wise.
Jérôme Laban 00:05:50 Now, if you go down a little bit, then there’s the access to the platform APIs. Let’s say, if you want to do talked about your location before. So if you want to do geolocation, if you go for the lowest common denominator, then you may have access to parts of the APIs that are, that the underlying system provides, or you may have to design your APIs yourself and try to do the abstraction yourself. And if you do that, then you may have some issues when you’re adding a third platform for instance which has a different set of APIs. And we are talking again about geolocation. One of that specific part of geolocation is user consent for accessing the geolocation. So that’s kind of things that need to be thought of in terms of API and choosing whether you want to reuse or use a cross platform framework or develop your application directly on the system, using what the platform provider suggests you use, whether it’s the web or iOS, Android or any other like Windows.
Jeff Doolittle 00:06:44 Yeah, absolutely. And this leads to what so many of these conversations, I think end up pointing to, which is that even something like cross platform user interface frameworks, of course, we’re going to talk about UNO and, and others today. Those are tools. And if you run for the tool without asking some of these broader questions, then you can blame the tool. But I would suggest you shouldn’t blame the tool because it’s really a design question. You’ve mentioned in some of your answer before, it’s sort of like, here’s the idioms of this platform versus what are the idioms say of our design and our product team and which end of the spectrum do you want to be there? To what extent are we okay with more generalized layouts maybe versus being pixel perfect. And again, that’s going to depend on really what you’re building. If it’s an enterprise application, you’ll make one set of choices, but if it’s a game you’re going to make different choices.
Jérôme Laban 00:07:39 Exactly. Game is because game is another interesting scenario and it’s very different. I don’t know, we don’t go into the gaming specifically because it has its own set of very specific constraints. But with regards to the choices that you need to make in terms of you’re getting your application there, running on all those platforms, I think it’s all about reuse and making sure that you don’t spend cycles just repeating the same, I would say mistakes, but it’s the bad side of the coin, but repeating the same process and having different set of customers that asking different things. And then the interpretation of what’s the, the designers and whichever architect type of person may be there to decide what’s there. And that may or may not have a familiarity with the platforms that are being discussed with the overall architecture.
Jérôme Laban 00:08:25 So let’s say if you’re doing a simple application or a very, that has a vertical, that it’s quite simple, in a sense, let’s say you, Uber. You could probably do something that’s fairly explicit for each platform because you want the experience to be fine-tuned to the maximum extent possible for the platform. And that’s probably fine if you’re spending, let’s say five times the money on developing an application there, but not everyone’s able to do this, unfortunately. And new application development in general is expensive and is getting more expensive by the month that are passing by for a variety of reasons. So it doesn’t make sense to waste inner energy in re-developing things, particularly if your core business is not necessarily the UI, but everything that’s in between and developing all this may be very, very expensive.
Jeff Doolittle 00:09:21 Absolutely. And it’s interesting, you say it’s getting more expensive, because most things in technology we’re used to them getting less expensive over time. And yet it seems that this is a space where, where we’re wrestling with that and hopefully tools like what we’re going to be talking about today can be part of the answer to, some solutions there for companies where it makes sense. So maybe digging a little deeper into some of these technical and organizational issues that make cross-platform applications uniquely difficult. It’s one thing to say, I want this device to run on Android and iOS. It’s another thing to say. I want to have the same app run on Android, iOS and web and desktop. Oh, by the way, Linux desktop two, please. And I guess I’d start with, is that even a good idea?
Jérôme Laban 00:10:02 Exactly.
Jeff Doolittle 00:10:03 I think the answer is maybe some it’s the consultant answer, right? It depends. But talk with us a little bit about that challenge and then maybe we could dig into a couple others.
Jérôme Laban 00:10:11 The first question that comes up is, for most people that are trying to do this and we’ve been trying to, I’ve been at a developer for mobile apps for a very long time. Let’s say something like 11, 12 years now. So it takes me back a bit and trying to do that kind of reuse or cross platform and things like that. And what do you do when application can span from a, let’s say I’m going to take the extreme, but a smart watch to a hundred inches screen that’s touch for some reason. How do you make an application that resizes that much and scales across all those screens? So if I trim down a little bit the question is, I have my app running on the web and then I have my app running on mobile. How do I make my UI react between those two? The non-good question is kill the UI? That’s not the good thing to do.
Jeff Doolittle 00:11:05 Especially down to your watch from a hundred-inch screen. Yeah.
Jérôme Laban 00:11:08 Exactly. So that doesn’t work. So then, then you can do things like readjust your UI. Let’s say, instead of being your recent tool, you go vertical and then you can change things like removing columns, removing data, have burger menus that expand and resize, but then you have issues like let’s say you’re an iPad for instance, and then an iPad. And even in normal apps you don’t even have to go to a large screen. Let’s say your iPad allows you to put things on the site. I think itís called Snap or something on iOS and changed names multiple times. But anyways, the idea is that your application’s opened. You have navigated to something, let’s say you’re in a, let’s say master detail view, then you click on your master. Then you go to the detail, then you’re in full view.
Jérôme Laban 00:11:54 Then you take your app and put it on the side. Suddenly you only have a third of the size for your app. What do you do? Do you show the details? Do you show the master? If you’re in the details, if you go back, what does it do? If you’re in the master, but then you selected something, it has some UI choose with it, then what do you do? What do you change? So that’s the of design consideration to have on UX to take into account when designing your application. And in many cases, people decide to just not support that because they don’t have the budget for it. And they say, my app doesn’t resize or it supports desktop and it’s UI rich or data rich. And I don’t care if it goes down to a mobile sized device, the other way around is I care only about mobile and form factor.
Jérôme Laban 00:12:46 It can resize rescale a little bit because of these sizes, the screen and things like that. But if you put it on, let’s say an iPad, it’s going to be, it’s going to look awful in most cases or they’re just going to shrink everything and put it in the middle. So that’s the kind of consideration that you need to have when you’re trying to get an application to run across multiple platforms. And then it goes, if you’re talking about very large screen, then touch, then we can have, we in UNO, always report console and put devices, so it can be remote control. It can be mouse, a touch, a pen and things like that, many things that can influence the development of your application and it can get quite difficult to get there. So that’s the taste of things that need to be considered when going across platforms.
Jeff Doolittle 00:13:29 Yeah. And I imagine even with the tools that can allow you to do cross platform, all the things on every platform, you still have to make the design decision about whether that is the right thing to do. Maybe it makes sense to do a desktop and web. And even though the tool might allow you to do mobile, you might decide that we’re not going to use the tool to do that for now, but that’s, that’s going to be up to you as, as a team. It’s just, sometimes people see a tool and it’s like the Swiss army knife that has 50 different things on it and they want to use them all. It’s like, well, all we really need is these three. So, let’s just do this and we can do this. So I’m curious in your experience though, if that kind of desire to use everything about the tool, sometimes bites people as well in this space.
Jérôme Laban 00:14:11 Well, money comes back always. So yes, ultimately let’s say the developer or the architect or whichever the someone that’s in that position’s going to say, I want to target everything because we don’t want to have anyone left behind. But the problem is that new development costs, even if the tooling allows you to do cross-platform development, you still have to test on every single platform to make sure that it works properly, and in many scenarios. So, the one that comes back often is accessibility. And for regulatory purposes, in many cases you need to have that enabled and it may behave differently. In some cases, in your platform your framework may not be able to handle all the scenarios that you want to have, or maybe visually it’s not doing the right thing. Or maybe some platforms do support suspending your application or killing your application in ways you can expect.
Jérôme Laban 00:15:05 And let’s say your analytics on Windows, your application’s going to run as long as it’s not closed. But unless you turn on explicitly, but it’s hard for user to do that. But on iOS and Android, I mean, the application can be stopped at any time because the OS decides so. So that’s the kind of things that need to be thought of when doing your development and going across all those targets can be difficult. So, the framework, the underlying framework tries to help you there with providing all the hooks in a cross-platform way. But it’s not always possible because sometimes they don’t exist or they’re not available, or they don’t map to anything that’s cross-platform in a way.
Jeff Doolittle 00:15:41 Yeah, absolutely. And now we’re getting sort of some of the behind-the-scenes things as well having done a lot of mobile development myself and my career, and just, you’re mentioning some of the life cycling differences between say iOS and Android, and even the ways they handle, when are we going to background an application? And then when are we going to basically shut it down, but then also what mechanisms are we going to allow for caching state so that if you restart the app, it looks like it didn’t shut down. So that to the user, it looks like it’s seamless experience, but you’re saving resources and battery and this sort of thing. But the way they do it is different too. So that’s where I imagine these Cross Platform tools. It’s not just about the user experience and the user interface. It’s also about how you manage these different life cycling patterns and other things. And what else might there be in that regard that the team might struggle with? There’s the life cycling thing, but what else, as far as platform specificity, have you seen, be things that teams need to consider?
Jérôme Laban 00:16:40 Well, a lot of them, we can go from memory management to CPR architectures. The weirdest one that I got was some Android devices that I have at heterogeneous CPU. So some cores go fast, some others don’t, and then it can do weird stuff because you’re having that kind of thing. So if you assume that your API is able to do some things at a certain speed because you’re doing it that way, then if you don’t have a CPU affinity, then you’re going to have stuff like that looks like that. Or you can have all sorts of touch, speed issues or animation issues because your devices of a certain style, or you can have local issues on, let’s say a specific variant of a specific device that run on a specific country, behaves a certain way then no other device does.
Jérôme Laban 00:17:24 So that’s the kind of things that can be happening when you’re developing. So, it adds up to all the things that need to be considered when testing your devices across platform, even if you’re using the same code that runs everywhere. There’s keyboard input, can you actually use a hack? A actual keyboard, physical keyboard, mouse input. Can you use a mouse on an iOS device? Yes, you can. And if you can, then that means that you’re going to be able to do things that you cannot do with a finger. So things like this, if you have a mouse, that means that you can have a hover and hover states on a button and needs to do specific things. So it’s not something that you add after the fact, if you have a keyboard, that means you have focus.
Jérôme Laban 00:18:04 So that means you have to have focus hints. Then you hit tab multiple times, then which control are you on? And if you’re only on a touch device, you can’t do focus because youíre on a control or you’re not. So that’s the kind of things that makes it different. And then there are platforms that don’t have threading. For instance, if you go on web assembly threading is very, very new. And in some cases you can do threads or you can’t do file system access or you can’t do lots of things that you can or can’t do. And you have to take into account the fact that it may not be available when you’re developing your application. So, the last one that I got was, I’m on the web and I want to open a PDF file in a PDF viewing application that’s on my device. How do I do that? The browsers don’t allow you to do this. You have to download the file, then open it. That’s kind of things that you can do on the mobile phone, you can do on a desktop, using a native app, but you can’t do on the web. So, user flow a different, depending on the target that you’re on.
Jeff Doolittle 00:19:01 As you’re talking about, all of these heterogeneous approaches to things and all the various things we would want to do in these kinds of applications, it occurs to me too, how important it is for a design team to understand the capabilities of the platform that you’re targeting and that you’re dealing with. Can a cross platform framework like UNO help with that kind of aspect of the conversation with design teams? And if so, how have you seen that work
Jérôme Laban 00:19:28 In terms of getting access to the features of the platform, UNO is based on the UWP API. So, the two parts of UWP, which is not supposed to be used anymore. Now it’s UWP is trying to get to win UI, but we can get into the detail. But the idea is that we have two parts, the first part of WinRT, it’s a set of APIs that have been developed since Windows 8. So, working on IOT devices up to large surface wall things, I don’t remember the names, so the idea is that all those APIs cover the absence or presence of things available on those devices. And that means that because UNO is implementing those APIs and providing access to those and across the platform way, we’re able to, let’s say, provide a way for developers to know what’s possible and whatís not possible.
Jérôme Laban 00:20:17 And then make that bubble up to the design to say, you’re on that platform that is not possible. So, we have to find a way to surface that somehow, so let’s say you want to share a file, is it possible on that platform or not? Do we show with the button, do we say that we change the behavior to say, send by email or use a server site component to do something, considerations in that sense? So, in terms of what we know is trying to provide is trying to give access to as much as the underlying features of the platform, based on what Microsoft already thought of in terms of API design, when they designed it originally for Windows 8, Windows phone and others, because they had that breadth of devices at the time. Kind of reaping the benefits of this in a sense, because we’re implementing those APIs and we’re talking about the screen resizing and things like that.
Jérôme Laban 00:21:09 They thought about using what are called the adaptive triggers. So, let’s say you trigger your UI to say, when the screen size is that length, then do that thing. If it’s less than that, then do the other thing. If it’s on a low power device, then do that thing. And if it’s on the web, do that other thing, visually, that’s kind of things that is possible. So, you can go down to specifics, but then overall it’s mostly about determining from your design perspective. What can I do on the set of platforms that I want to target? And it’s not because UNO targets all the platforms that you can see that you have to target all those platforms at once, because it may not be possible or money efficient for you to target all those platforms, considering the state of APIs that you want to use. That’s the biggest question that you have to ask yourself when developing and designing your application
Jeff Doolittle 00:22:02 Now, to be clear, you mentioned UWP, which I believe is Universal Windows Platform, is that correct? And just to be clear, you’re describing UNO platform when you talk about those APIs being implemented, but you’re not saying that because it uses Universal Windows Platform or WinRT APIs that you are building a Windows or WinRT enable or a Windows only application, it’s not at all you found. And your and we can talk a little bit more about this, as the show continues, about the different approaches that people have taken and the teams have taken to solving this issue of cross platform UI. But to summarize this, I think what I’m hearing you say is the UNO platform team looked at the APIs that Microsoft created for the Universal Windows Platform and said, these are good APIs, and they’re even more generalizable beyond just Windows. And so you’ve adopted those APIs really as an abstraction, but then you’ve created your own implementation around those abstractions that allows you now to deliver for Linux, Android, iOS, Windows and web.
Jérôme Laban 00:23:10 That’s exactly that. And the important part is that we don’t have to design anything specifically because designing APIs is very difficult. When people they spent, I don’t know how much they spent, but it must have been painful to design all this. And the idea behind all this is to say, yes, it’s called UWP at that point, previously UWP. And now we split it into it’s WinRT APIs and Win UI APIs. So, Win UI is just all about showing things on the screen and WinRT is all about manipulating the system. So, file access network, sharing things, opening files, saving files, things like that. So that’s were any kind of sensors that you want to get access to.
Jeff Doolittle 00:23:52 All the IOs stuff? Right.
Jérôme Laban 00:23:53 Pretty much, pretty much the iO, yes. iO and system access in general. And those APIs were well defined. And what we did is basically extract the APIs for this and incrementally implement those APIs as we go in depending on what our clients are asking when going on target hub. So that’s the idea for this and, and yes, it’s definitely an abstraction. So it has its advantages and disadvantages. If you are making abstraction, then you may be hiding things from the developer. So that’s why generally we do provide access to the underlying APIs that we are using as well. And that’s what the .NET team has been doing with original Zamarin. And now continuing onto .NET6 is that it’s not an abstraction in terms of your developing with C#, but you get access to all the APIs that are available underneath.
Jeff Doolittle 00:25:44 Yeah. And it makes sense the power outlet in your wall is a great abstraction. On rare occasions you need to get around that abstraction, but we really hope you know what youíre doing when you do that.
Jérôme Laban 00:25:54 Exactly.
Jeff Doolittle 00:25:58 And so these abstractions have been adopted here and that sounds like that’s some aspects of that I imagine are similar across the different cross-platform framework, but let’s maybe step back and, and take a little walk down memory lane and give us some context for how we’ve arrived at where we are, because cross platform UI, in some sense, I think has felt like a Holy Grail to people for quite a while. And I believe the iPhone came out in either ëO7 or ëO8. And that’s really when smartphones became actually smart. I mean, I had a little pocket PC phone and then it was definitely, I would not call it a smartphone. I’d call it a dumb phone with a little bit of touch screen and a little stylus. But since then, things have definitely transformed significantly. So you and I both been in this space for a long time and I’d love to just hear from you, what’s gone well and what have been maybe spectacular struggles or failures along the way. And then that can lead into now, how are those things being addressed now?
Jérôme Laban 00:26:57 Well, so much has changed. The iPhone changed quite a bit and by pushing some technologies out and forcing some in. The most spectacular one was, were actually two, in the same space where Silverlight and Flash. Those two were pushed out because of the iPhone and iOS was not allowing for many reasons and for good reasons that the rest of the industry took in itís wing later on, but was the security aspect of it in performance considerations. And those two, what kind of an attempt at doing Cross Platform UI in a sense, because you were able to run your Flash, I wouldn’t say mostly everywhere, but at least you were able to do on Linux to some extent then Mac and Windows. So that’s the three major one that they know were there at the time, but still it was rendered UI from most of the content.
Jérôme Laban 00:28:42 So it’s kind of a very wide range of things and because of that diversity, and if Microsoft failed, particularly as a result of this and because of the cost, we were in the middle of this for quite a while, when developing Windows 1 applications was the fact that people were saying, now we do have iOS and Android, or they have their flaws, but they still work. And there’s, there’s a community and there’s a lot of users around it. Amazon failed in that, in that respect in the same way, but you’re trying to get your applications run on something else, than iOS and Android, then you get banned down again to the cost. Do I want to maintain my applications on an additional device and have a set of developers that don’t do the same things and know it comes down to just everyone trying to enhance and get their platform to be available somehow aside the big ones and gets difficult. So it’s repeating history in a sense.
Jeff Doolittle 00:29:39 Yeah. And back to what you said before about it getting more expensive, it makes sense because you alluded to the fact in the quote, unquote ìearly daysî, that what we were dealing with is how do I run this on multiple operating systems? But you’re pretty much dealing with two chip sets back then you got power PC and you got Intel, AMD which is the same X86 architecture. So they’re pretty much dealing with two and then beyond that, what you’re really contending with now is the different operating system implementations on top of that architecture. But there were only so many screen sizes. There were only so many CPU speeds. There were only so many this and that. And now you get into all the different form factors that have their own way of doing things and the operating systems on them, which aren’t always as unique as maybe they could be. Although there are efforts with things like Pinecone and Framework laptops to sort of get us back to more user control over those sorts of things. And I’ll put links to the Show Notes to those projects. But all that to sum up and say, no wonder, it’s getting more complex. We’re trying to do more with more things. And those more things keep changing and it keeps throwing these new stuff at us. And so previous attempts to do things like Flash or Silverlight, or you remember the old Java Applets apps?
Jérôme Laban 00:30:51 Java Applets, yeah.
Jeff Doolittle 00:30:51 Yeah and the Java Applets would spin, and it would say ìLoading Javaî and that sort of thing. And yet ironically it’s Java that runs Android, but it’s different than it was then.
Jérôme Laban 00:31:02 Itís just the language, we’re talking about the UI, the distinctive UI, of Java Applets at the time.
Jeff Doolittle 00:31:07 Oh yes, distinctive is a good way of putting it. Yes. Yeah, so there’s been a lot of approaches. So let’s maybe shift now and say, all right, with all of that, to some extent, the more things change, the more they stay the same, we just have more complexity to deal with now. So what are some of the different approaches to Cross Platform UI? You’ve mentioned a couple, Microsoft has Xamarin, which is moving towards something called MAUI. We’ve talked about UNO a little bit. What else is out there and what kind of characterizes them and differentiates some of those different approaches?
Jérôme Laban 00:31:38 Well, let’s say there are three ways for rendering things. So we were talking about the UI specifically, there are three ways to do it pretty much. If we’re talking specifically about plus platform things. So there are frameworks that do pretty much take the UI from the underlying system and lay it out in such a way that you can use Cross Platform tooling to lay out your native items in the way you want. So that means that if you want to display a checkbox, a text box, a button or something like that, then all of this is going to be looking like what the UI provides underneath. And then if you go on another system, it’s going to be laid out closely to the same, but no may not be exactly the same. So you may need to go into some length to line everything properly. And then your designers going to be saying, is that checkbox?
Jérôme Laban 00:32:31 And they’re not looking the same because at that location, they wanted this. Then it looks differently when running on iOS or on the web, things like that. So there’s the kind of things that you’re getting there. But the advantage of that kind of technology is that, in terms of rendering, it costs not that much because pretty much relying on what the system is able to do and to provide you. And if let’s say iOS 17 has a brand new 3D way of holographically displaying your, items on your screen then, it’s going to be there. So I don’t know about anything there, but saying that following what the system provides is quite important. We went from secure more for them, to metro-ish designs, to flat layout and things like that. So that’s kind of things that, that needs to happen.
Jérôme Laban 00:33:18 And then if you do that, that means that you’re really following closely what the system provides. So the most important example specifically is when you’re in text box and you press long press on the text on iOS, there’s a little magnifier that happens to show up on top of what you’re trying to see. So you can read it better and that comes for free because you don’t need to do anything. You’re using the native controls to do that. So that’s, some of the frameworks are doing that kind of rendering. So we’re talking about the UI layer. We’re going to talk about the underlying afterward. But then there’s the other way, the completely opposite of the spectrum. The other one is draw everything and you treat the screen as a bunch of pixels? Which means that basically when you’re going to be rendering, your application is going to be looking the same across platforms.
Jérôme Laban 00:34:05 You’ll be down to the pixel. The animation’s going to be the same. The response to the user input is going to be the same. So it’s going to be for kind of a Nirvana for your designers, because it’s going to be very nice for cross platforms. The problem with that approach is, that your users are going to say, well, that’s not a native application. So it may be to some point of the space is going to say, well, that’s a designer consideration users don’t care if their application look native or not, but there are power users that are used to specific behaviors of controls. And then if they’re not behaving the same way for any kind of reason, because we know Apple fine tunes everything down to the microseconds because they want their controls to behave in a very specific way.
Jérôme Laban 00:34:54 For good reasons most of the time, then if you’re drawing everything yourself, then what you can do is just mimic what happens on the platforms that you’re trying to target. So that means that you have to follow. That means you have to implement multiple versions of it. You have to implement it’s a, the Cupertino way of displaying things and then the material way of doing things. And the material V3 version of things. So that’s lots of incrementals for this. So kind of a problem in a sense, because you can do lots of stuff with this, and then you can go down to some devices that don’t even have an underlying system. So for instance, you go on an IOT device. There’s probably no UI tool kit in there at all. So if you draw everything to yourself, then you’re good to go. And that kind of devices, because the only thing that you need is pointer XY location.
Jérôme Laban 00:35:42 When you touch the screen and then the rest is going to be the same as running on a shiny device that is very expensive on the other end. So that’s where kind of use the best tool for the job is kind of interesting. And then there other frameworks that do in between, in between the two, which is kind of use the controls on the platform or use the UI toolkit from the platform, but Chrome less in a sense. So, let’s say if you want to have a text box, then show the cursor for the text box, but then don’t show anything else. That means that you benefit from everything that has touch input or things like that, and the interactions from the system untouched. But then if you want to have a video text box, you can do that because that’s possible because the framework is going to be re-rendering everything for you and you don’t have to do anything there. Now that approach has also its advantages and disadvantages. Being in the middle like this means that there may be things that are possible in one sense, or maybe there’s performance drawbacks. So it’s kind of a choose what’s best for your scenario because in some cases, rendering, everything may make sense on the other end rendering nothing yourself may make sense, just layouts and then doing in between may make sense because your scenario makes more sense. So that’s the baseline in terms of what’s happening with the UI part of all this.
Jeff Doolittle 00:37:09 Yeah. And I mean, that’s a lot, right? Also the one you choose is going to have some impact. So this is one of those areas where when you’re about to make an irreversible decision is it’s good to make sure you’ve spent the time to ponder these things because choosing one of these frameworks is going to have implications for your teams for the long haul for sure. As far as the more hybrid approaches that you mentioned, you didn’t use that word, but between the two extremes of fully native versus draw it all yourself and somewhere in between, other things come to mind, Flutter is one of them, UNO platform is one, are there others? And with those, can you maybe help our listeners understand maybe what distinguishes some of those different frameworks that are trying to solve some of these same problems?
Jérôme Laban 00:37:56 You have many names around. So maybe your react native is on one side, you have Maui and Xamarin forums. You have the react that runs on the web. That’s close to react native, but not really. Then you have Blazer on the Microsoft side that does not yet cross platform, but is going to be with Maui. Uno platform, which does all of the above, depending on the platform, there are many others like Avalonia, that does kind of a rendering that uses all pixel rendering. So choosing follow platforms depends on two things. The first one is, what do you want to do with your UI in terms of rendering? Do you want to have access to everything by the pixel, or do you want to have access to the system? So, in terms of the UI, so that’s kind of top level choice in terms of what you want to do or, and in between, but with trade off, depending on what you’re trying to get to, then the underlying would be, what’s the language that you want to use.
Jérôme Laban 00:39:39 And then you have to have C++ Interop. And then you do, you go to C++ and then, or Java interop or things like that. So you can just go directly to the platform. So it’s going, what’s best for you in terms of, do you need to interop a lot with the system, if you do, maybe choose one that allows you to do without too much nuts and bolts to change and tweaks to make, and you have to factor all this in to make the choice, to determine what kind of target you want to use. At UNO we wanted to get to some point where we’re able to get access to everything. And with regards to the UI that we’re trying to get to is, depending on the target, the either we render everything or we use a hybrid approach, and you let’s say, if you want to run an IOT device, then UNO allows you to render everything by a pixel. But if you run an iOS and Android devices, then it makes more sense because you may want to have access for user to do accessibility. And that’s where having access to the hybrid approach allows you to do this, but still allow you design to be in an because they can design whatever they want. So that’s the kind of things that we’re trying to get to UNO. So that’s pretty much the landscape in terms of very high level, at least.
Jeff Doolittle 00:40:46 Yeah. And that’s again, helpful to kind of get that broad overview. There’s so many things going on here that it seems like in some ways, these tools, they might reduce some complexities, but they also can introduce other ones. Which complexities would you rather deal with?
Jérôme Laban 00:41:06 Pick your poison.
Jeff Doolittle 00:41:07 Yeah. Like you mentioned, design Nirvana and I think product Nirvana is just, it runs anywhere and it looks consistent and it’s like, what do those words mean?
Jérôme Laban 00:41:19 It depends on who is it consistent for? And I was alluding this to the fact that people are saying, oh, I don’t want my design to be the same everywhere because I, as a user, don’t want to have to deal with the UI. That doesn’t look the same because I want to use the controls that have been designed by Apple. But who am I to say this? Am I representing my users? Maybe not because my users don’t care and what they want is an application that works. So, you design your applications for your users and not necessarily for the power users that designs the applications. So that’s the questions to ask is what’s your audience. If your audience is set of developers, maybe you want to do something that uses the native APIs or the native controls, because maybe they’re going to be akin to find those differences and kind of bash you in a way, when they’re doing things or because they expect some system to behave a specific way.
Jérôme Laban 00:42:08 Now, if you’re running on a, I don’t know, banking application, maybe banking is different, but letís say any kind of application end user application. And you want your users to be at home when they move devices, which is likely to be the case. They’re going to be using it from their mobile phone when they’re moving and then coming back to the web and then expect the visual cues to be the same, because they don’t want to get lost in the application. So that’s kind of things to consider as well. So, where you, the developers may not be lost in that regard because they used to the kind of differences between, and they’re used to get to the differences between platforms where other user may not. So, who’s your user basically?
Jeff Doolittle 00:42:45 Yeah. I know your audience, who’s your user. And thinking about particular tools comes to mind. I mean, one of the ones that I’ve always thought did a really good job of consistency is Trello. And there’s just a ton of consistency between how it functions in the different IDMs and the different form factors. And yet that’s very much leaning towards the pixel perfect side of things, but something else I notice about that, and other solutions is when you start getting away from more of the atomic controls, like check boxes and text boxes, and you start getting to richer controls that are composed and now have more behavior that are involved in them as well. I imagine that’s going to draw you to some design decisions about where you end up on that spectrum of very pixel perfect versus taking what the system already provides for you.
Jérôme Laban 00:43:36 That’s correct. And a composibility is another thing. Being able to get your controls to do exactly what you want is something that’s very powerful and being able to create your 3D video, enhanced button with flashy colors that your designer wanted, is something that may not be easy to do with all platforms. And if your intent is to have this, then choose wisely because not everything allows you to do that kind of things.
Jeff Doolittle 00:44:04 Sure. And you said something, I find funny, you said our control should let us control how things work. And ironically, sometimes our controls feel like they do the opposite. They feel like they’re taking control away from me. So it’s just
Jérôme Laban 00:44:18 Exactly. Yeah, it is. Design is hard. You design a UX. It’s very difficult to get to, and especially across platforms, because it may not make sense if you’re designing something for touch and then going to the web, on the desktop, then it doesn’t make sense. Some touch interactions don’t make sense at all. Kind of like the most prominent example is hover stuff. You’re hovering onto something because you want to expand or add more content and it makes sense you hide stuff away and Apple’s been permanently good and bad at doing this because even on Mac OS, if you want to try to get a hold onto the scroll bar with your mouse, because you want to go down one pixel or because whatever you want to, it’s very difficult to get to. And it makes sense on the web, on a touch device, because you can touch the screen, then move. And then the scroll bar, you don’t care about the scroll bar, but if you want to touch screen, and then for some reason you don’t, you can’t scroll with your two fingers because of whichever IDM youíre on, then don’t move those things away. So it’s kind of a difficult way to get to this and those UXs depending on the platform need to be thought of and thought through properly, otherwise you’re going to get your user lost.
Jeff Doolittle 00:45:29 Yeah. And it’s amazing the ways that can couple, the things that we don’t even realize, the very fact that we have on mouse fill in the blank means that for our entire careers, we have coupled events to a specific solution, which is mouse. And yet really it’s about user interaction is what you’re after. And now with things like touch versus hover versus these things, you see that even those then are coupled in interesting ways to design decisions about the form factor and the idioms of the device that you’re using. Then you start to wonder, and we shouldn’t go here, because it’ll take us far afield, but just kind of a thought exercise for the listener is then what is the right abstraction? Or maybe the right’s the wrong, what is a broader abstraction, because that’s really what we’re hunting for when we talk about this is, okay this is a useful abstraction and that’s fine, but it’s also leaky and coupled in some interesting ways. So what’s the next abstraction that we are looking for?
Jérôme Laban 00:46:28 It always comes back to the scenarios for users and for UNO specifically, recently, most people that are coming to us are saying, well, I want a desktop windows application that runs on the web as well. Or I want a mobile application that can sometimes run on the web, but mobile only because I, I don’t want to publish to the stores, things like that. And getting to designed for those specific subsets of the, the supported APIs supported platforms makes more sense in some cases to get there. And I was talking before with the abstraction and things like that, or kind of supported APIs, it’s the focus ability of controls. It’s something that you don’t really need when you’re on a mobile device. And then when you go back to a desktop, then you need that. And if your framework doesn’t support this, because it was mainly developed for mobile, then you may have a problem when, when your user start just tapping through to get to the controls that they want to and leak or not. You’re going to have to support that somehow, because there are people that are only use the keyboard and they want to use a mouse or did, because they can’t use, they cannot use a mouse. So that’s the very important aspects of, of all this. So lots of consideration when choosing a framework.
Jeff Doolittle 00:47:38 Absolutely. Well, let’s dive a little more deep then into UNO, specifically having kind of look more broadly. We’ve talked a bit about Xamarin and Maui and Flutter react native. I donít know if we mentioned electron, but that’s another one that comes to mind as well.
Jérôme Laban 00:47:52 Yeah. Well, we could talk about that one. It’s actually quite interesting because of the wazon (?) space, but with regards to UNO, what we’ve been trying to get to is to get to C# and XAML across platform. With reducing the burden of designing the UI of an application and developing the UI of application across all those platforms that we mentioned. Web assembly being the biggest driver lately either because it’s the newest one or because it’s the farthest reaching one or the future proof one, I don’t know.
Jeff Doolittle 00:48:23 That remains to be seen. That’s the promise.
Jérôme Laban 00:48:26 Yeah, it’s the promise, but still it’s getting interesting and a lot of things are happening in that space. The W3C and the implementations that are happening in the browsers are making this go a lot faster lately with regards to all the features that they’re adding. They added exception handling and SIMD for vectorization for computation. They added everything that goes along with running web SIMD outside of the browser with interface types and things like that. So lots of things are happening in that space. And that makes for very interesting things, like we have a client that’s basically ported his own application from, I think it was 12 or 15 years above C development. I’ll remember exactly how much, but the very, very large application that now runs in the web and kind of the investment and reuse, itís just massive.
Jérôme Laban 00:49:11 And they intend to go to other platforms though the promise of that application is very heavy. Desktop applications have a tendency of being data crammed. I want to have as much data as I can. And that’s completely the opposite of what mobile apps need to be doing, which is I’m showing on what’s needed to be seen. And that’s what makes those applications a bit complex to go from the desktop to the mobile, even building the other way around is a bit easier. Let’s say you have to, all those considerations, so that’s the kind of thing. So what SIMD been driving quite a bit from the past three years and it’s still moving. And the good thing is that with regards to developing on all those platforms is that we’re trying to get to the widest really reaching audience in terms of the APIs that we’re trying to implement as well as the number of platforms and performance that we’re getting there.
Jérôme Laban 00:50:06 So we’re trying, we’re investing a lot into what Linux is able to do with regards to the CIA backends that we’re doing as well as on iOS and Android, getting as many features possible implemented there. So still using all the same C# and .NET APIs and with all the things that the Maui team is doing underneath so we’re, we’re using everything .NET is providing and Maui is sitting on top of this and we’re sitting in the same thing. The Maui is doing is using .NET for mobile, for iOS, Android, Mac catalyst, and Mac iOS. And we’re pretty much using the same API. So we’re standing on the shoulders of the.NET team for that specific part of the support we’re having for mobile. So kind of aiming at a wide range of supported platforms, but still using .NET and C# and XAML everywhere. So that’s what we’re trying to do. It’s kind of a large summary of what we’re doing.
Jeff Doolittle 00:50:56 Yeah and you’re keeping Linux desktop on the table. Oh, desktop table, sorry. Excuse the pun. But because my understanding is that Linux desktop is not going to be supported by Maui but, platform is continuing to enable delivery of a Linux desktop application as well.
Jérôme Laban 00:51:12 Yeah, definitely. I mean, we’re getting a lot of requests for getting things to run on Linux for a variety of reasons, whether it’s because people do like C# and .NET and they want to be able to develop on their targets because they don’t want to run on Windows or because they have specialized hardware with regards to licensing and things like that. They want to run there or because they have specialized, IOT devices and they want to run in that direction, and the reason we’re able to do this is because the way we’ve been rendering our UI is by using another internal abstraction. Well, actually it’s a public abstraction, but it’s called the Composition API? And Microsoft has been using this APIs to render the XAML UI on top of this. And we basically reimplemented that part of the API on top of Skia so Skia is a framework to render pretty much pixels in a sense. And then we’re sitting on top of Skia, using Composition API, and that Composition API allows us to do drawing images, shapes, texts with sprites, clipping, things like that all sorts of things that you can find in a retained mode API that renders pixels that allows us to render things across platforms and specifically on Skia and everything up top. Like if you and render a button, that button is going to be rendering the same way across platforms using that kind of APIs.
Jeff Doolittle 00:52:34 Yeah, that’s great. And I think if it’s not clear to listeners yet, but it seems clear, we just want to make sure when we’re talking about UNO platform, while we’re using XAML in this space as an abstraction over UI, and we’re using C# to build our application, these applications not only are targeted for multiple platforms, meaning they’re not locked to Microsoft or Windows or things of that nature, but it’s also possible to develop these applications. If you have a Mac or a Linux box or a Windows box it’s possible for you to do your development, and you can target these platforms from multiple places.
Jérôme Laban 00:53:07 Exactly. It’s all about getting C# and XAML across platform. So, starting maybe from Windows if you want, but it’s not necessarily a requirement. And then you can develop from your Linux box, if you want using VS code or you can use GitHub code spaces if you want, or Gitpod if you want as well. So that’s all the kind of development environment that are available. We know four that were released a few months ago. We added support for VS code using a new intel and hot reload support inside of VS code so you can be helped writing your XAML inside of visuals to your code inside of GITpod or GitHub code spaces if you want. That’s the kind of scenario that makes it interesting.
Jeff Doolittle 00:53:46 Yeah, that’s really great. And speaking of that, if any listeners out there are using GitHub, if you haven’t yet found, if you’re on a Mac it’s command period. If you’re on a Linux or Windows it’s control period, it will open up something very cool. I will leave you to do that yourself. And you will thank me later if you did not know about it. So just control or command period, when you’re in a GitHub, looking at code in a repository, it’s pretty cool. So back to UNO, this sounds really fascinating. And this isn’t specifically about the platform itself, but you’ve mentioned a couple times that you have clients. So I guess there’s a business model here with UNO platform. So maybe you can speak about the relationship between the UNO platform itself, and then what keeps UNO viable, I imagine is that there’s a business model that allows you to keep building it out. And that’s probably good for listeners to know that there’s something driving this too, that it’s not just going to disappear tomorrow. So, speak to that some if you will.
Jérôme Laban 00:54:44 We’re basically using the RedHat model, which is hire us to fix the bugs that you’ve opened pretty much if I sum it up in that way. So, the idea is that you can go on target hub and open issues and if there’s someone that’s in the community and wants to fix it up, great, or you may want to fix it up, that’s good and make a contribution and we can review the PR. But then if you don’t have the ability to do that kind of contribution, then you can contact us. And then we’re going to be taking a look at those issues, using a paid commitment. And basically, what happens is that your bug or issue or feature request is going to be bumped up at our priority lists based on the clients that we have currently so that we can fix the issues that you’ve raised. That’s the way we’re sustaining ourselves. So we have multiple clients. We have our parent company nventive, which also does that, which uses UNO platform. So we have a lot of applications that have been developed using UNO and still being developed using it. So we’re making lots of changes inside of UNO and adding features and things like that based on that. So that’s basically the model is. Let us know what you need, open issues, and then contact us if you need those issues to be fixed faster.
Jeff Doolittle 00:55:55 That’s great. So you’re using it for applications that you’re building for customers, therefore you’re building the tools that you need to do that and improving them accordingly. And then for those in the community who want to use it, they can use it because it’s free in Open Source. And if they find a gap, I imagine you accept pull requests under certain contribution guidelines. And if they don’t want to send you a pull request, but they want something to be up in the priority list, there’s a way that they can go about that as well.
Jérôme Laban 00:56:23 That’s exactly it. So of course, accepting contributions is always something new specific, because when you accept a contribution, then you make it your own. So it’s not because you’ve opened a pull request that we may accept everything. And that’s the same thing with regards to handling Open Source and this sustainability model of every Open Source project. If you open a pull request, because you done something it’s not just your time that you have to consider. It’s the time, the maintainer that needs to be looking at your PR, making it part of the environment in a proper way, and then live with it once it’s merged in.
Jeff Doolittle 00:56:58 And we know that maintenance is far more expensive than implementation by order of magnitude, at least.
Jérôme Laban 00:57:04 Exactly. And if you take a look at the way, cross platform, frameworks work and especially UNO because of the number of targets that we do support is that when you are creating a feature, it has to work everywhere. And that makes it complicated in terms of support. So that’s why we’ve been segmenting everything so that developers can provide partial implementations. So, let’s say you can implement an API for Android only if you want, that is possible. We have a way to indicate to developer that API that you’re trying to use, it’s not available on the targets you want to run. So that’s the kind of things that are available and you can do that at runtime, if you want to. That’s the kind of, of things that are possible. So, let’s say it’s not available in specific version of UNO, and then you update it, then it becomes available. Then you can feature flag it and say, now it’s available. I can use it. So again, it kind of thinks that are possible. But anyways, the point being that accepting a PR is a commitment.
Jeff Doolittle 00:58:00 It is no drive by pull requests.
Jérôme Laban 00:58:02 Well I mean, you can do a drive by pull request but it doesn’t mean that we are going to accept it.
Jeff Doolittle 00:58:06 Right. And that’s the idea more generally with Open Source, get to know the community, get to know the contributors, earn some trust, maybe make some small things, but don’t just say, here’s this massive pull request and I’ve moved everything around and changed everything and here, take it.
Jérôme Laban 00:58:22 Yeah, exactly. So that doesn’t work and it’s always the of getting to what do I choose first? There’s that nice PR that’s been implemented for a while and that there’s another client that’s paid for something. So how do I place priority wise those? And I’m sure the communities are based in the same way that are treating their background in the same way, using pay clients and Open Source contributors and you having to deal with that kind of priorities. It’s tough in some cases. So, we’re glad to have a model that works for us, but it’s not always possible for everyone to kind of offer paid support. Cause if someone pays you to do something, you have to do it. So, you can’t just accept monthly the money and then do the, not do anything afterwards. So the ecosystem needs to change a bit in terms of getting, we’ve seen that with Mt. Fuji and the number of emails that have been sent to the maintainers requesting and demanding fixes or Come on people it’s Open Source stuff. You can’t expect all this that’s not possible. And the .NET community has been different in that sense, say, we can discuss that one if you want to, if we have the time, but it’s been a bit different in terms of expecting things around where Microsoft has been the provider of things, which is not the case in other communities.
Jeff Doolittle 00:59:36 And has been shifting bit by bit over the last 15 years which I think is, has been a good thing. I do wish we had more time to explore that, but we don’t. I will say though, you’ve proven one of my things, which I’ve been kicking around as a theory going forward, which is from now on no podcast is complete until there’s a reference to Log4j. So thank you for proving that. And we should just end it right there because that means the podcast is over. So for those who aren’t familiar, we’ll put a link into the show notes for Log4j, although I will wonder where you have been for the last three months, if you don’t know what the Log4j world, but yes, Open Source generally. And there’s some great stuff to explore there.
Jeff Doolittle 01:00:13 Maybe in a future conversation, we can dig a little bit more into Open Source more generally, but as far as now, I know in my research on UNO platform, there’s some other integrations you mentioned VS code. I know there’s an integration with Figma, which is a design tool that designers use and to get closer fidelity between that design Nirvana and the actual implementation is a reason, obviously that you’re delivering that sort of a solution. What’s next for UNO platform? What are some big things you’re working on, say over the next three to six months?
Jérôme Laban 01:00:45 Improving performance, improving platform support, like new Linux support, MacOS support, .NET6 support, because Microsoft is moving quite fast in that direction. So getting all that support, correct, improving, .NET7 for WebAssembly is going to be there with lots of support for performs in an OT for web assembly, newer features like we hope threading to that’s how things that is going to keep us busy. The Figma is going to keep us busy as well. Figma support where you can design your things in Figma and then export that into your app to get it running without any clicks and navigation included, things like that. So yeah, lots of things that are going to happen during the year that’s for sure.
Jeff Doolittle 01:01:23 Great. Well, if our listeners want to find out more about what you’re up to, where can they find you? On Twitter or GitHub?
Jérôme Laban 01:01:29 Twitter or GitHub? So @UNOplatform, if I remember correctly, then we get the two whats there. We’re on Discord and GitHub discussions also, if they want to ask questions and to get started, you get a platform.uno/getting started. If I remember correctly, mostly Twitter and Discord too, to discuss things around.
Jeff Doolittle 01:01:47 And then for you personally, on Twitter?
Jérôme Laban 01:01:49 I’m on Twitter, so J L A B A N search by name, youíre going to find me there and be happy to discuss cross that of things.
Jeff Doolittle 01:01:57 Absolutely. Well, it’s been wonderful having you here. It’s clear that there’s a breadth of experience behind everything and your passion is evident. So, I appreciate you two giving us kind of a broad swath of Cross Platform UI generally, but we were also able to dive in a little bit more into what UNO platform is doing specifically. So really glad you joined me here today on the show.
Jérôme Laban 01:02:17 Of course, thanks for having me.
Jeff Doolittle 01:02:18 Okay. This is Jeff Doolittle for Software Engineering Radio. Thanks so much for listening. [End of Audio]