Episode 555: Herb and ReactionView with Marco Roth

DAVID HILL Happy Wednesday and welcome to the show. I'm recording at Rails World today in the Buzzsprout booth and joining me today is Marco Roth. Welcome to the show, Marco. Thank you much for having me, David. And this might have been a coincidence of just kind of when the podcast kind of went on hiatus for a while with the timing of when you kind of came up with Stimulus LSP and Herb and then ReactionView, but you've never been on the podcast before.

MARCO ROTH No, I think that's right. I was supposed to be on the podcast when Elise was hosting it still.

DAVID HILL Okay. But I think it never happened,

MARCO ROTH But I think it never happened, so happy to be on it now.

DAVID HILL I'm very happy that I get to correct this kind of error in history where, like, you've done so much recently with all of these developer tooling pieces that you've built and put together and made possible for us to all use. I feel like there's just, like, a huge oversight on the podcast for never having you on the show.

MARCO ROTH Yeah, happy to be here to talk about all the tools I've been working on.

DAVID HILL All right, so let's get into the... big thing that really made me feel like, okay, now's the time because I can start the conversation with something relatively new instead of pulling all the way back to Stimulus LSP or Herb when it was originally put forward. Relatively recently, the Herb engine was merged into Rails main.

MARCO ROTH That's right, yeah. So I guess as time of recording, this was happening at the end of August, two weeks ago. And it's really exciting to have it in Rails because The way ERB views have been working this whole time is actually really cool because eRuby, the previous engine that was the default for all templates, has been serving our needs for the longest time. A very long time. If you look at the code, actually, eRuby is such a novel piece of a gem. It's like a few hundred lines, maybe 200 lines, and it just does all of it. And if you look at the comparison, I guess, between eRuby and Herb... There's so much more complexity in Herb, but it also does so much more, which is the kind of things we have been missing, I feel like, for the view layer in Rails and all the tooling around it to make it a nice experience to work in it that we have been missing. And that's kind of what Herb is enabling now that we have all these things. And Herb is still good. It's going to be still used for anything that's not HTML, HTML templates in Rails. Right.

DAVID HILL Right.

MARCO ROTH But going forward in Rails 8 .2, any HTML templates is going to use Herb any fault.

DAVID HILL So is that going to result in any particular changes for developers in the actual development process?

MARCO ROTH In the sense that you have the same tools in your editor that are checking your syntax, doing the syntax highlighting, giving you the autocompletes, that's now the same parser that's running in the compiler, in the engine, that's compiling your views in Rails. So this is a similar kind of story to how Prism kind of came about. You were able to parse Ruby with Prism. And now Ruby itself, the actual runtime, is also using Prism on the hood to compile all the Ruby codes into instruction sequences. And this is now the similar story, I guess, for HTML ERB, where you use the parser now, which is pairing all the tooling, and now the Herb engine is using the same parser to compile all your templates. Okay. Yeah.

DAVID HILL Okay. Yeah. So it's just the Herb engine, though. So it's just kind of like the foundational piece of... The ecosystem of tooling that you've already put together, like there's more of Herb beyond that, there's the reaction view beyond that, is the goal to eventually try to build all of that directly into Rails 2 on top of the Herb engine that's now in there?

MARCO ROTH Yeah, so the whole thing is that over the years as I've been working with Hotwire, I've noticed that there are some kind of things that still feel kind of odd.

DAVID HILL Just a few.

MARCO ROTH In general, it's a really good idea, the way it works. It's really powerful if you know how to use these tools and if you're not afraid to write some JavaScript. And I think, actually, Jeremy gave a talk today at Railsful where he kind of showed all of these things that can lead to problems if you don't kind of know how to deal with them.

DAVID HILL Which was a great talk, by the way. It was a really great talk, yeah.

MARCO ROTH way. It was a really great talk, yeah. And I guess he was on a previous episode to talk about this. It was a really good talk. And it just kind of... validated the point I was kind of trying to get at with this whole Herb and Reaction News story, that there are so many kind of issues that are not right there on the surface when you kind of start looking at it. But as soon as you get into it a bit deeper, you notice that there are so many things that could go wrong. And I just thought, like, if you have this powerful tooling now, that was one of the first ideas I noticed when I was working on Herb. I was like, actually, we can do this whole engine after that too, right? actually built this whole reactivity into it. If the engine and the parcel understands all the things you're doing in the view, you can do more advanced things in the compile step when you are looking at these templates. And that's what I'm trying to be looking at and how Herb can power these more ambitious and more dynamic and reactive UIs that you weren't really able to do with BotWipe before. Or you would have to have a lot of JavaScript to make that kind of work and make it seamless.

DAVID HILL I guess most Rails people don't like to write JavaScript,

MARCO ROTH like to write JavaScript, so if they can avoid it at all costs, then this is, I guess, the kind of thing they want to look at going forward.

DAVID HILL I've done a lot of work over the years in React and with Vue, but I've always loved Stimulus just because it actively tries to be very minimal JavaScript. It's usually not focused on manipulating the DOM the way React and Vue are. JavaScript sprinkles the way DHH usually phrases it. It's just like little tiny pieces.

MARCO ROTH just like little tiny pieces. Just like the behavior, right? You don't try to render markup, you're just trying to add behavior to existing markup. Because if you do it that way and you keep the markup rendering back on the server,

DAVID HILL if you do it that way and you keep the markup rendering back on the server, I just felt like all of the idioms that I needed to use to accomplish something... I'm always still thinking in the typical Rails idioms of how to accomplish something instead of having to map, okay, I got this thing from Rails, and now I need to do it the React way, which is a very different way, and mount it up on the front end and glue it all together.

MARCO ROTH is a very

DAVID HILL I just hated all of that.

MARCO ROTH Yeah, I mean, I've been doing some React over the years too, and one of the nice things about React is that you have to write your components in React, obviously, but the way you kind of have the state being managed by React itself, You don't have to go in and add a triple stream to say, this ID on this page needs to be replaced with this new content. You just tell it, this is the new state, and React itself figures out how to reflect these changes, what it has to re -render, what it has to look at and fetch more data. And that's kind of one of the powerful things about React, I think, is kind of undervalued in what we have been doing in Potwire, because you have to specifically say, This part of the page in the TurboFrame, I want to have this part reactive or refreshable. And then I have to go and expose a different endpoint if it's on the same page or look at these matching IDs so the TurboFrames match. Otherwise, you get the content missing. And if that doesn't work for you or you want to break out of the frames, you have to go and use streams. But again, this is kind of all you have to write this code to say. I want to rerun this part and update that part of the page. But it's all kind of manual that you have to do that. Right. And that's what I was kind of trying to get at with this whole reaction view, is to have the compiler of the Herb engine figure out all these dependencies statically, like using static analysis, to figure out what actually is this view doing and how can it's reflect the changes. So you just set state again, like React does, and it knows if this state changes, it has to kind of look at these parts of the page and then only re -ending those. That kind of makes it so much more easier to think about dynamic UIs because you don't have to think about anymore, oh, I have to put this data controller here to put the Stims controller there. Then I have to wire up this Stims controller. It has to fetch new content from the backend. But then you have to re -render some of the partials because you want to add new content into it. But you have to kind of wire up the stimulus control to your backend and being able to render HTML all the time. And you were doing this over and over. And if you want to build more ambitious UIs to kind of have them feel modern, you'd have to re -render a lot of things. That's kind of what I was trying to look at. If I can make this possible using all this analysis now that Herb can provide, how we can make that happen.

DAVID HILL You're speaking here at Rails World tomorrow.

MARCO ROTH That's right, yeah.

DAVID HILL Are there any surprises that you can tell me that won't make it to the podcast until next week?

MARCO ROTH Yeah, so I guess since this is going to air after the talk happened already, tomorrow at Rails World I will announce a new version of Herb, which powers all of the reactiveness in ReactionView. So ReactionView is going to be available from tomorrow, second day of Rails World. Nice. And it will make any view view you have today reactive out of the box without having you to write anything else than just having your existing news.

DAVID HILL Nice.

DAVID HILL So just for the sake of context, what do you mean by the view will be reactive?

MARCO ROTH So when you kind of try to update a value on the page, it can refetch the values from the backends without you having to go in and say, I want to scope this just to this part of the page. I don't need to wrap this in a turbo frame. I just say I update this new value. The kind of client runtime in the browser knows, oh, this value changed. So I go and refetch that value. And because ReactionView using Herb is able to figure out which parts are dynamic, which parts are static, it can build this dependency tree of all these states that are dependent on each other and try to refetch whenever something changes. So you have... The initial page render is still all regular HTML over the wire. But we have this new format, which I've been calling slots. This just means you have addressable parts of your template on the client side that is addressable. So you can say, on slot number one of this template, I have this new value for it. But the client itself has this kind of map of where these dynamic values are. So it can go in and update these values dynamically when they arrive at a later point. Either from a subsequent request, be it over action cable or be it over server -set events, or even using client -state optimistic UI, it can provide these new values and tell the runtime to re -render the view using these new dynamic values. And since it has all the static content already on the page, it can client -state render your views on the client -site. Oh, wow.

DAVID HILL Oh, wow.

MARCO ROTH So this is kind of similar to how, if you have been following Phoenix Live View does it, But it actually takes it a step further, so you actually can client -side render these templates. So you have the existing ERB templates, but they can be client -side rendered on the client, because the compiler splits it up and pairs it in a way, so the client can take them, store them on the client side, and then it just needs a new dynamic value that's coming in from somewhere, which is usually just a JSON request. Wow. And then it can update and re -evaluate that stuff, which means... There's no DOM morphing involved. There's no turbo frames, no turbo trips involved anymore. It just knows where dynamic values are and can reflect these changes dynamically. And that's what they call in the JavaScript world, fine -grained reactivity. That's what this enables.

DAVID HILL Wow. I can see that being a really big change in just like how development feels of just having that out of the box.

MARCO ROTH Right. And that was kind of the design goal to just make it look like IRB, right? So you know and use IRB already today. to write your views. But why can't you use that kind of logic to also write your dynamic content, right? We don't have to use JavaScript to do that. And that's what I was trying to get at. Just keep your existing views as they are today and just make them more powerful because we have all this analysis in the background that can figure out what you're doing and then provide this kind of intelligence on top of it. And that's something that we weren't really able to do before because we didn't have the whole herb parse on the analysis available. Right.

DAVID HILL Right.

MARCO ROTH But this kind of picks up from stimulus reflex almost like six, seven years ago. And this kind of was popular before Holtweiler came around. We were trying to do these similar things that's what reactionary does today. But back then, we didn't really have this herb parse available. There were some important tooling pieces we didn't have yet.

DAVID HILL There were some important tooling pieces we didn't have yet. And herb wasn't able to exist because we didn't have prism yet.

MARCO ROTH didn't have yet. And herb wasn't able to exist because we didn't have prism yet. So this kind of dependency chain kind of is resolved now. And I finally kind of... was able to put all these ideas out of Stimulus Reflex into ReactionView. So ReactionView is kind of what I always wanted Stimulus Reflex to be. And this is now kind of releasing tomorrow. Okay. Yeah.

DAVID HILL Yeah. So were you part of the Stimulus Reflex team? Yes. When that was a thing?

MARCO ROTH Yes. When that was a thing? Yeah. So Nate Hopkins started, I think, back in 2018. Okay. And I was looking for a tool that I wanted to use for building these dynamic UIs. Right.

DAVID HILL Right.

MARCO ROTH I used it, I found some bugs, and I was like, oh, I have some more ideas that can kind of improve this framework. And then I was invited to help maintain the stimulus reflex and capability libraries. But then after that kind of was kind of popular, Hotwire came out like a year or two later. That took kind of the whole hype from stimulus reflex away,

DAVID HILL That took

MARCO ROTH from stimulus reflex away, and everything went into Hotwire. And we had to revaluate what we wanted to do with it, because it wasn't really competing. Because it solves different parts in different ways, like how you build these dynamic UIs. There was some overlap,

DAVID HILL There was some overlap, but it was like a Venn diagram. The circles connected, but they didn't completely overlap.

MARCO ROTH was like

MARCO ROTH Right. Then after the frames came and after the morphing came, also in turbo, it kind of was more narrow and more narrow, which you're like, it doesn't make sense for us to... build this alternative next to Hotwire. So we kind of took the idea and were like, yeah, let's just try to make Hotwire itself better and keep soft -deprecated stimulus reflex, but try to focus on Hotwire itself. And I guess it didn't really work out in the way that we were hoping to kind of bring these ideas back into Hotwire, which was then why I was trying to figure out what I can do about it, which is then how... The stimulus LSP and triple LSP came about. I wanted to build just tools around it to make it more enjoyable to work with these tools. And that's where the whole need for Perp came about. Yeah, and that's pretty much how this led to now ReactionView, which kind of ties back all the way to the stimulus reflex days, which I think is now the most natural way how you write dynamic UIs and Rails.

DAVID HILL I didn't know you were part of the stimulus reflex team from back in the day. There's... kind of this kind of interesting progression, I guess, of just like those tools, like you mentioned before, like all these things that needed to fall into place. Yeah. From Stimulus Reflex to the Stimulus LSP and the Turbo LSP and Prism and all of that finally like, oh, now you can finally do this thing that you've always been imagining.

MARCO ROTH Yeah.

MARCO ROTH Right, yeah. And it feels really cool. I was at home working on my talk and was like trying to build demos to actually feel how it, yeah, how it feels. What can I do with it? How far can we push this idea? And I was kind of blown away myself a few times. Like, oh, this is working. This is actually working almost too well. It feels like almost wrong, but it really changed something inside of me. It's like, yeah, this is, I think, how I always expect this to be. Or how the Rails should have been from the beginning.

DAVID HILL Well, that's a good sign if it's surprising and impressing even the person who wrote it.

MARCO ROTH Yeah, and I guess the whole design idea is that... I have been seeing a few companies over the years trying to look into Hotwire for their own applications, and they were like, yeah, we can make this work, but also there are all these other tools in React and other frameworks which you can just use, and they will just work. And if Hotwire starts to hit the wall at some point, and then it's really hard to get from there to scale it up in a way. Rails has been known for... scale the framework really well in all aspects. So if you need more databases, multi -databases, if you need the sharding, you can scale up your Rails app really well. But I feel like for the view layer, for the frontends, there wasn't really this progression from, I just created Rails app with ERB, and I want to get to the point where I have fully dynamic SPA. This is like this whole spectrum. There's nothing in between, right? And there's no progression of how you can go from one side to the other if you ever need to. So the idea was for reaction read, you also kind of build that gap to try to make it so you can progress from one side to the other or back if you want to and have that kind of coexist in the same app and kind of make it so you can go step by step, page by page, controller by controller to kind of move things over. And because it's all kind of gradual, how you can adopt it, you can use some pages. You can use an X2. stimulus and turbo it just works in the same way and you have an adapter for stimulus too so you can control all the state and how stuff is being re -rendered so there's a JavaScript API under the hood too you can use yourself if you want to build even more ambitious things which aren't really covered by the new declarative kind of markup right that reaction now offers so this kind of should give teams the kind of one ramp to go from one side to the other and kind of scale up their front end if they ever need to

DAVID HILL I'm getting a little excited about this, honestly. I haven't had the opportunity to work with Stimulus or Hotwire in general for a long time, but just a little personal project related to podcasting that I've started putting together with AI in the last couple of weeks and getting to work with Hotwire a little bit in there. It's like, oh, I should install Herb. I don't think I have that in here yet. And so, yeah, I think I'm going to have to pull that in and see what it does for me because this sounds like magic.

MARCO ROTH Yeah, and I think it's kind of, I got a few questions over the past few months, like, why are you investing in these tools if you have agents write that code anyway, right? Like, why do you care about developer experience at all? Can I write code manually anymore? But I feel like the more and more these agents take over our workflows, I think you actually want to have a tool that understands your views deeply and can tell you if something is wrong and what is wrong about it and can pinpoint these errors in your templates on line numbers and column numbers to tell the agents something about the code you just produced is wrong.

DAVID HILL Right.

MARCO ROTH And to give the agent kind of a feedback loop to validate its output. And that, I think, these guardrails is what makes these agents powerful. And if you don't have any tools to validate what it produces, it's kind of tricky to get good code out of these agents. Right. I think Herb is perfect for that because it can power all these workflows and you can write lint rules. It has a bunch of lint rules built in itself where it can tell something is not right or wrong about it.

DAVID HILL Right.

MARCO ROTH And just having this validation step as a concept, as part of your CI and your agent's workflow, it should catch more of these errors and bad practices in your apps.

DAVID HILL apps. Even on top of that, right before we came in here to record, I think we were both in Rachel Wright -Munn's talk, where she was talking about generators and the work that she did for Ruby events, which I guess is another thing we could talk about. We could talk about Ruby events for a little bit here if we wanted to. She was talking about how the whole sidetrack she went down with the MCP possibility with generators there. Using MCPs really only works if you've got the external tools that they can call to do that work that they haven't specialized and focused in. So those tools still matter for the AIs even because if you're interacting with an MCP process to get something done,

MARCO ROTH So those

DAVID HILL it's going to be better if it has a tool that it can do it. that it can call to do that work.

MARCO ROTH that work. Just check its output, right? What it produces is actually sound. Right.

DAVID HILL What it

DAVID HILL There's a lot of interesting things we're going to have to pay attention to with Herb and ReactionView in the future, it sounds like.

MARCO ROTH Yeah, and the other thing I kind of was able to do now, because we have this whole reactive layer built into Herb, we have this dev server locally now that whenever you change your views, it will instantly update in the browser without having to reload the page. And again, I think even if you are having your agent work on your views and just having the browser kind of reflect the changes as the agent is working on it, without having you to go in and refresh the page every other second to see what it just produced. You can follow along and see live what's happening. I feel like another nice addition that doesn't seem to be talked about a lot is just give introspection of what the agent is currently working on so you can steer it in one way or another. tell it's going the wrong way right which you know they still do from time to time yeah it doesn't seem like they are perfect but i think having these foundational tools like herb like rubocop like prism like or even at the ruby decks that's kind of came out a few weeks ago they give the agent i guess the structure and i guess the backbone to yeah about the output and i think that's what we need to

DAVID HILL which you know they still do from time to time yeah

MARCO ROTH it doesn't seem like they are perfect but i think having these foundational tools like herb like rubocop like prism like or even at the ruby decks that's kind of came out a few weeks ago they give the agent i guess the structure and i guess the backbone to yeah about the output and i think that's what we need to focus on more to just give the agents more tools if you really want that future to be in the future, right, of coding, if that's where it's going.

DAVID HILL Yeah. I feel like I'm a relative latecomer to actually trying real agentic coding and code generation. I really started giving it a try with this podcast -related app I've been playing with. So having an AI kind of really generate a lot of the code, but I started from a... Chris Oliver's Jumpstart Pro template. I started with that, and with that as the framework and the guardrails for the application, I felt like the AI was paying a lot of attention to the things that were already there. It had explicit instructions almost to not duplicate features and functionality that were already there in this mostly empty application template. But there are a lot of features that are already... recognized as like, oh, I can do that over there already. I don't need to build something from scratch for it. And it just felt like I could give it maybe slightly more vague instructions. It's like, I want this next. Because it was already following some guardrails of things that the application was already doing, the ways that it was already doing it. It felt like it was going smoother than a lot of the horror stories I've heard of it just going completely off the rails. It's like, oh, it already had guardrails that were keeping it on the path they wanted it on. Ruby events is one of the few open source contributions that I've actively tried to make in recent years, just because I have had a hard time historically finding open source projects that I have a certain minimum amount of interest in and feel like I could contribute to. But then after meeting Rachel and talking with her at, I think it was RubyConf earlier this year, it was so easy to just go to Ruby events and it's like, oh, well, there's several issues here on the issues page for adding certain pieces of data to certain events. I can do that. Let's go look at that.

MARCO ROTH Yeah, I think that's the kind of problem with open sources. There's so much to do, but if you just come in and just want to do something quick, that's... Usually that's how it works, right? You have to kind of, you yourself have to invest some time to understand the project. What is it trying to do? And how can I help and be involved in?

DAVID HILL involved in?

MARCO ROTH But I feel like with Ruby events or like conferences in general, you have some connection point to it already. So you already know if you have attended a conference, there's a schedule, there's usually a call for paper, there's usually speakers and the details about the talks and all of that. So this is already familiar to you. So you kind of know the domain. intuitively already. So if you just have up -to -date information, you can just make a contribution and that just feels more rewarding because you achieved something, right? If it's your first contribution, you contribute to open source for the first time and it wasn't something that's super hard in that sense, but just going through this whole process and this workflow of contributing to an open source project is something that could kick off a career in open source and make other tools better. Lots of people have gone that direction.

DAVID HILL people have gone that direction. Other people, myself among them, I've had a hard time finding open source contribution opportunities in the past. And so Ruby events, it was a special thing for me. It was like, oh, wow, open source can be easy. That's weird. I don't think I've experienced that before.

MARCO ROTH And yeah, it's in the end, it's just that for the data, it's just YAML files. But still, it's something you can relate to and give back in that sense to make a contribution. I think that's really valuable to have. Yeah. And even though we are trying to automate a lot of these things for LLMs, just to kind of make our lives easy, because there's so many events happening, and it takes a lot of time. You would have to do this all by hand. And you're not even going in just one direction.

DAVID HILL you're not even going in just one direction. Like, you're not just going with future events. You're constantly trying to, like, get information for past events, too. Yeah.

MARCO ROTH So I feel like just having this archive and this index and this knowledge of what the Ruby community is talking about. You can probably do some more analysis on that data and figure out trends, figure out maybe even new talk ideas or figure out what it wasn't talked about before. Like when I try to work on new talks like this one tomorrow, I like to just go in and search for keywords and see what was talked about 10 years ago. Because I know from indexing some of these lightning talks and some of these older events that there were so many good ideas like in lightning talks, like birded lightning talks. And because they're usually like this hour, two hour long video on YouTube that doesn't have any timestamps, we'd never kind of stumble upon that if it wasn't like categorized and structurally indexed in Ruby bins. Right.

DAVID HILL Right.

MARCO ROTH But now you kind of can just search for them and you can find them and see if somebody was sharing a gem before at a conference or did a blog post about it or had some kind of other, was sharing some other kind of piece of content that you wouldn't be finding otherwise. And sometimes, people had different ideas because it was just a simple time. And I found quite a few inspirational things from these old conferences, which is kind of cool.

DAVID HILL Interesting. I should visit Ruby events more often than I do these days, just to do some of that, to just pick through old talks and old conferences and see what kind of things people were talking about. It's fascinating. Also like seeing Matt's 10,

MARCO ROTH It's fascinating. Also like seeing Matt's 10, 15 years ago, talk about what's currently happening in Ruby for like Ruby to something. Oh man.

DAVID HILL man. It's always kind of fun to just have this like different,

MARCO ROTH man. It's always kind of fun to just have this like different, I guess, view of the Ruby world. Oh, yeah. Going back in time. Because the world was so different then.

DAVID HILL Oh, yeah. Going back in time. Because the world was so different then. Cool. Is there anything else you would like to talk about before we close up?

MARCO ROTH Yeah, I would just like to invite everybody that has been struggling with Hotwire or trying to not use Hotwire in the past. Like, try to give Herb a go and try to give ReactionView a go. It's designed to be drop -in. And if it's not drop -in, then that's considered a bug. So the more we can kind of have... people try out in their applications, you can make it better for everyone. And it seems like there's some momentum right now to kind of keep pushing and make this view layer story in Rails more nicer for people and agents. So I feel like if we can do more of that, we will have a bit of time writing new features and applications.

DAVID HILL I love it. Thanks for joining me today, Marco.

MARCO ROTH Thank you for having me and happy Rails world.

DAVID HILL Enjoy the rest of Rails world tomorrow. I'm hoping to record a little bit more before I'm done here today. This has been the Ruby on Rails podcast. I had a great time chatting with Marco today. Thank you to AppSignal for sponsoring this podcast. And thank you for listening.