EP. 264 Beyond the Database: Mastering Multi-Cloud Data, AI Automation & Integration (feat. Peter Ngai, SnapLogic)

0:06Hello everyone and welcome to another MongoDB podcast live. It's great to have you with us. I'm Shay McAllister and today we've got a fantastic show as ever lined up for you. We're diving into the one of the most exciting shifts in modern data architecture, the move towards multi cloud, hybrid and edge computing. And that's where Snap Logic comes in. So it joining me today is Peter, whose principal architect at Snap Logic. And Peter is going to breakdown how Snap Logic is revolutionizing the way businesses connect data across clouds on Prem and beyond. We're going to talk about the challenges that enterprises face in multi cloud and hybrid environments and of course, how Mongo DB and Snap Logic work together. And we're going to get a first hand look at the Snap Logic UI and exclusive showcase of Snap Logic's latest AI powered data automation in action.

1:01Peter, you're very welcome to the show. How are you? Hey, how are you? Thanks, Shane. Thanks for having me on. Not at all. It's great to have you. So where are you joining us from, Peter? I asked our guests, our viewers, to say where they are coming from. Where are you based? Sure thing. I'm based in the San Francisco Bay Area, so more specifically if you're familiar with the area, I'm I'm actually located in the East Bay. That's where I live. Snap logic offices are actually located on the peninsula, so that would be over to the to the West side of the Bay. So, but currently I'm I'm home so. Excellent.

1:35So we're getting you at somewhat the start of your day, tail end of my day here in Ireland. And I know the US daylight savings changed at the weekend right too. Let's keep it current. So I think the scheduling of this gave me a little bit of angst kind of going on my on my 5 or 8 hours on my 7 hours. What's what's the difference? That's right, that's right, yeah. And usually with the first week of daylight savings, it's a little, it's a little bit sleepy, right?

2:02Because like you lose an hour right in the spring of sleep and, and, and it just takes me about a week to adjust to it. So my kids, it takes any longer. But nonetheless, it's it's all good. Yeah, Yeah. Well, listen them it's great to have you join us, Peter, as like, look, I, I talked about some of the things that we're going to dive into in the introduction, but one of the key things that I'm always interested in, I know that our audience are as well too, is for any of the guests I have.

2:31I love to know your path to date, your career path to date. You know, how did you end up here? What were you doing beforehand? What did you do all the way back in college? You know, where have you been since etcetera that took you to right now with your current role as architect and Snap logic? Yeah, sure thing. Well, I graduated from RIT. That stands for Rochester Institute of Technology. That's located in Rochester, NY. That was quite a few centuries ago, but I do hail from the New York City area in tri-state area where I was born and raised.

3:07And so naturally after graduating college, I stayed up in Rochester. And if if anybody's familiar with that city, there's, there's basically like Xerox and Kodak were the were the major companies there. So I weren't there for a number of years for both companies and then moved back home to New York City, worked on essentially Wall Street for a number of years as an independent. Then eventually it came time to, to raise a family. My wife and I wanted to raise a family.

3:36So we wanted to move out to find a place in, in the country where it's a little bit different. So we, we came out here, backpacked out to, you know, the San Francisco Bay Area and just kind of hunkered down and, and we've been here ever since about 2005. And so since I've worked at a lot of, you know, I worked at some Fang companies and essentially where I used to work prior in Fortune 500 companies, now I find myself working for startups seems to be all the thrill in the Bay Area. So that's how I wound up with Snap Logic. Excellent.

4:11You're bringing your Fortune 500 knowledge back to the startups basically then. So these are how the big companies do it. This is what you, you, you need to be thinking about as well, too. It's interesting that you mentioned Rochester. I, I, as I said, I'm from Ireland. I've been traveling to the US quite a lot for work over the years, but I'd say it was probably 20 odd years ago. The last time I was in Rochester, I used to work for an actual data storage company, a RAID systems company, and we used to do our and what what they used to call accelerated life testing of our systems there.

4:45So they had a testing centre there that was unique, that could do earthquake testing, all of this kind of high frequency life, half life failures as well. So I, I spent a few, few trips to Rochester, but that again, long, long time ago. Perfect. So you mentioned then culminating in snap logic. Umm, tell us a little bit for the viewers who don't necessarily know, tell us a little bit of, you know, around Snap Logic, a kind of a high level overview of what Snap Logic does, what the problem it's solving for its users and in the kind of modern data ecosystem that we're in.

5:24Sure thing, sure thing. So essentially Snap Logic is an eye pass platform, right? So it's an integration platform as a service. And what does that essentially mean is that we help connect data sources in a company's ecosystem one to another, also applying transformations in between. In addition to data, we also integrate applications across as well, right? So that's one advantage. So for example, if you wanted to connect your, you know, Mongo database to like Salesforce upon certain events that would certainly happen, then you can certainly do that. And the advantage of snap logic is that it's a basically a code free or a load code platform, right? So it's ease of integration there. OK, so you're kind of like an interoperator between all of these platforms and, and obviously, you know, I think particularly in large enterprises, Mongo DB included as well too.

6:30We, we use so many different platforms as well. It's like, well, how do we connect all these things together? And, and obviously from our own perspective, we have integrations with our key cloud partners and many other tech partners as well too. But you're sitting in the middle space then Peter and Snap logic, you have to be almost agnostic. You just need to make sure you can get, you know, from A to C through snap logic being to be in the middle, right? That's right.

6:56That's right. And it should be like a seamless interaction too. And that's where it gets somewhat tricky at times, right? So you have a lot of different compliance space that changes a bit. So you're talking about different environments as well. So it's not just like point A to point B, it's like you really have to skip over like hurdles at certain times. Yeah, I can imagine it's hard, you know and as maybe and I don't know, but I would imagine your engineering team or very much on top of the moving shifting, you know ecosystem that's out there in terms of what people are building, what people are adding and you know what knock on effect it might have for snap logics infrastructure, right.

7:42Yeah, exactly. Well, it would be to a certain degree. I think a lot of it depends on how you build the system, right? Because it's a very natural problem space. I mean, if you look at engineering and you had to connect one data source to another, it's just a very natural inclination to like, I'm going to write that one script that does that, right? It's no one's paid job to really do it, first of all, right? So it's just someone says I can write that in 5 minutes and they do that. And the next thing you know, someone changes something like a, a login or a password script.

8:13I know what, I'm going to decouple my credentials into a separate data store. Well, that's another piece, right? And then, oh, the API change, oh, that's another piece, right? So on and so forth. So a lot of it really depends on how you designed or architected your system. And have to say, like App Logic has done that really well where it's built upon a composable architecture, right? There are there's always going to be moving spaces in, in technology.

8:42It's just a fact of life. And essentially, if you build it right, you don't really have to change too much, but you would need to append on to it, right? So it's built very well in that degree where it's efficient and it scales well. OK, so when you and I were chatting prior to to prepping for this stream that we're doing, we talked a lot about flexible data architecture and the kind of various, I suppose you would call them pitfalls, but also, you know various things to be concerned about as to how this.

9:15But the flexibility, the advantages that gives you, if you bear that in mind from the outset, talk a little bit about that, like around, you know, avoiding things like vendor locking and, you know, being compliance and being that resilience that you just talked about. That's right, that's right. And those are just some of the key things of building a flexible arc data architecture, right? Because in this day and age of, of and there's also going to be, you know, the honest truth is like economically, globally, we're all trying to save money, right? We're trying to be more efficient with what we have, right?

9:49And we're also at the same time, we're also trying to maximize what our expenses are, right? So there's gonna be a few key principles that we look there for building flexible data architecture, right? Is the first thing is that companies really and there's and just take a step back. There's been a natural evolution from data centers right to clouds, public hypervisors. And now we're, we're embarking on the next step, right? So where companies are starting to notice that, yeah, there's there's the, the big cloud providers like AWSGCP, Azure, so on and so forth.

10:27But no companies really want to be locked in to like any one of them, right? So they're trying to think about, oh, how do I be cross cloud now? How can I be cross cloud and on Prem or on the edge right as well? So there's those factors in there. There's also the the factor of compliance, right So talk. To me, a little bit about on the edge because I hear that a lot and I, I, I, I kind of know what it is because we, we, I looked after a product that we repurpose for edge stuff. But and as an edge database, talk to maybe the viewers who mightn't be familiar with that on the Edge might mean Peter. Sure thing.

11:05Umm, so, so on the edge is essentially, if you have it, umm, if you can think of that as a computing device that would live, let's say for example, in your home, a Wi-Fi router would be a perfect example, right? So that resides in your home, right? If for those that have like cable modems at home and such for Internet access, so inside your Wi-Fi router, there's, there's certain intelligence there. There's actually think of it as a small computing engine in there that able, it's able to perform some functions separate from the cloud, right?

11:39Because everything in the cloud you're, you're we're always thinking that we're just, you know, computing things in the cloud and getting input and output from our homes where it's like you took like a Wi-Fi router that's in your home, you actually have a small computing devices there as well. OK. So any devices that may be, you know, an IoT device for example may also be edge device as well like right Wi-Fi routers, little light switches, thermostat might be in your home. I'm trying to stay away from product names there, but. I get you.

12:17We, we, I know when we were building our edge database product, we were working with some of the the cellular providers. So they were saying, look, when your when your mobile device is making requests, you know, why are we taking that all the way through the network when sometimes the tower that received the request can actually know what to do with it, right? Exactly, Exactly. OK, And here's a here's a funny little and I don't hear this term too too often anymore, but like about five years ago, there used to be this term called fog computing and people like fog computing.

12:51What's that? If you take a little intelligence from the cloud and you bring it down to the edge, it's like a fog. That's a new one. That's a new one on me, but I love it. I love the analogy. I think that I think that's really good and it's great. If anyone else knows of any other analogies like that, throw them into the comments. For those that joined and missed me, please add any questions you might have for Peter as we make our way through the show.

13:17It's great to see the folks joining us, you know from Pakistan and Indonesia and Bangalore and Denver etcetera as well too. So please do get involved. When I asked you about age there, I had stopped you going into the compliance part of how you were explaining what, you know what's to be concerned about with flexible data architectures, Peter. So talk to us a little bit about compliance then. Sure thing. So there's Prior to being a SNAP logic, I used to work at a small a healthcare company. And so we also used data talking specifically about data in the compliance space in healthcare.

13:58It was imperative that we needed to store our data on Prem, which meant, you know, not in one of the public Capital Advisors essentially. So we actually had Mongo enterprise on premise, right. So that became very compliant as well. Whereas the rest of our computing was actually at the time on Azure, right. So it was one of the public hypervisors, whereas our data was actually on premise. So in industries such as like healthcare and also finance, there would be some compliance according to where the data is stored.

14:34Also issues of topics of, excuse me, data sovereignty as well. So there's certain data that are produced in specific areas or regions in the world that need to remain there as well. So there's other types of issues and topics that we that that would come into play as well. Yeah, OK. And being based in Europe, I'm particularly aware of the issues because we have a very fragmented approach to data sovereignty and compliance as well too. That causes a ton of headaches, business opportunities at the same time, but a ton of headaches for, for, for companies as a whole, you know, and I'm assuming with with regard to the compliance side of things.

15:16The other factor then in the whole flexible data architecture is just pure resiliency. That's why many people choose multi cloud in the 1st place, right? It's to, it's to have that resiliency either from a physical perspective, a geolocation perspective, etcetera as well, too, right, Peter? That's correct. That's correct. So a lot of companies are moving obviously depending on your requirements towards resiliency or, or low RTO, low RPO, like whether you have to be in different availability zones, right? That's one, one area of like resiliency. If that's good enough, then they just, they're just basically in the same cloud provider, but just in different availability zones.

15:58They have to be in separate regions. That's another requirement. And today there's even there's a desire to be in, in multiple clouds as well, right? So you have all those different factors at play. OK, OK. And so that all of those different factors at play are factors that really help Snap Logic's position in that decoupling of of the architecture that companies are kind of ending up with these days. And as you said in your earlier example, yes, it's fine. Somebody can write a piece of code that can do something and then they kind of, as you said, move chain, the login breaks and then they got to decouple that etcetera as well too. So obviously snap logic brings a ton of benefits. I would imagine you know, given the the huge decoupling, it's, it's also quicker to integrate, maybe easier to scale, etcetera, more business value.

16:56Talk to us a little bit about the kind of what you see, the feedback from your customers, how it's how it's working for them. Yeah, sure thing. And that's a really good point, right? Because it always, it's always seeds the idea whenever Snap Logic is invited into a company to help integrate their systems, right, always starts as a seed knowingly there. It always branches out because it becomes such an efficient way of, of performing this task that it, it starts to just grow because people, you know, companies find out that hey, I just need to connect to this system, to this system.

17:40I can write a script or I could just put together a small pipeline in 5 minutes with snap logic and it's done, you know, and it scales out really well and, and, and so that proliferates through different business usages and it becomes very efficient, right? Essentially the app integrations and data integrations inside of companies usually doesn't start as someone's like I use the term day job, but essentially it's it's someone kind of takes it on as an extra, you know, an extra function would want to try, but OK.

18:17OK. And I suppose that might kind of move us towards kind of where does Mongo DB fit into all of this? Is that as well then, Peter, you know, how does snap logic interact with Mongo DB? What does Snap logic use Mongo DB for? Sure, sure thing. So the main data store, well, it, it integrates with Mongo DB in two facets, right? The first one is that Mongo DB actually is the main data store, right? For for, for snap logic, right? So we chose in those sequels data store years ago to, to house our data models, essentially all our assets and operational metrics, all that is actually stored inside of our of our Mongo clusters, right? So that's the first thing.

19:03It's, it's acts as the backbone data store for Snap logic. And now we also find out that there's a large customer base that actually uses their Mongo data source that they would like to integrate with their other systems, applications or their other data source as well. So we we we have we we have items called snaps. I'm trying to like see how it can best explain this, but essentially. And I know for those tuning in, we're, we are going to, you're going to show this, we're going to open up the screen and share the snap logic UI and, and, and go to that.

19:40But that's intriguing though, because you're both a customer and a partner of Mongo DB's essentially. So you're using it internally for your own operations, but you've obviously seen then your customers have data there that needs to be connected to something else. So hence you're you're also leveraging us as a partner. Yes, that's correct. Perfect, perfect. That's great. So I think so you touched on snap packs. Do you want to talk a little bit about what those? Let's set all the terms out, but we're going to do show and tell, obviously in a few minutes, yeah. Yeah, yeah, sure thing.

20:15So. So if you think about how snap logic is used to integrate systems in a company, it's what is built in a workflow, which is called a pipeline, right? So if you think about that as one executionable series of steps that would comprise of a pipeline, let's say as simple as read data from a file. That's one step in the transform the data that's from the file into a separate, you know, a separate format. That would be another step. The third step is insert this data into Mongo.

20:54That'd be the third step. So that comprises the pipeline. Now each of the steps are also known as, as we call it a snap, right, Because it's OK, OK, so yeah, so each of the snaps are configurable as well. So it's it's pretty integratable. And as you can think about, there's hundreds of different snaps, right? So and obviously you can snap them together however you want, like like. I like it. And obviously then the term snap packs is a is a pre built sequence of snaps that you can leverage and is, you know, you know, obviously kind of as you see customers doing certain things that they require kind of going, Oh, maybe is, is it the case that you're looking at it going, maybe this is something that all of our customers could benefit on.

21:42Let's join all of these services up and give them a template. Yeah, well, we do have template pipelines as well. Snap pack is essentially a group of snaps. So for example, we have a Mongo snap pack. In there we can connect to different Mongo data sources, have different operations and such, right? OK, OK, excellent. So, umm, I think we, we talked obviously about those and obviously you know, all of the introductions you spoke about the, the, the increasing complexity that we live in, in a, in a kind of a multi cloud and, and hybrid world as well then too Peter.

22:24Umm, so I would imagine that then you know that complexity and the way snap logic goes towards solving that has just grown the business over time and and keeps growing it. But I suppose in a roundabout way I'm saying the complexity of today's environment is a benefit to snap logic because it it's screaming out for the snap logic solution applied to it, right? Yeah, yeah, exactly. And if you look at the pace of of technology evolution over the years, it's just increasingly faster, right? Every step.

23:01And we're all familiar with the latest evolution of AI. And, and if you look at the previous incantation to that, it was just, it just seems shorter and shorter and shorter and we're just exponentially going quicker and quicker and quicker. So, yeah, staying in front of that curve, right? It is certainly the question, right? Yes, yeah, yeah, I'm surprised we got the 26 odd minutes into the live stream before mentioning AI that usually it usually comes up a lot earlier for for on most of my guests How and I know we're going to see kind of an example of an agent creator later on, but how has AI affected Snap Logic's business and and kind of how it thinks about things as well too.

23:47What what changes has the AI brought about for the company as a whole? Well, AI has always been part of the fabric or the desire snap logic, even years before chapter 80 came out. Let's just say I think AI was coined as machine learning. We had a staff of data scientists that actually evolved snap logic and build upon it. So we actually had predictive learnings of pipelines at that time, even before chat ChatGPT. Yeah. When AI came and now when when introduction of ChatGPT came out, it became very conversational, right.

24:31With the interface of of artificial intelligence and, and, and, and the staying in front of that, essentially that's what snap logic has done, right. So, so essentially we become now we're not trying to be the LLM or be the artificial intelligence engine. We still want to be in that integration space, right. So we enable connectivity to to the LLM and ease that integration. And then there and I'll show show this in a demo video, but essentially how one achieves that very quickly in in a very low code environment.

25:14OK, OK, excellent. I'm looking, I'm looking forward to that. And again, I'm seeing the the hellos coming in, in the comments. It's great to have Andrew from Indonesia and folks from Bangalore and Pakistan and Philadelphia. Oscar, you're very welcome as well too. Umm, you know, it's great to have you join us. Do any questions you might have for Peter, please post them up there. We'd love to entertain them. So we talked a little bit about the multi cloud and the hybrid world, et cetera. And you know, how you're managing Snap via snap logic to connect data up with the applications and platforms that need it, et cetera.

25:49We mentioned compliance on the introductions. How with regards to like, I presume you're kind of, you know, my world is like an ETL, right? Extract and transform and load. That's the way I'm, I'm looking at some of this and I know it's, it's much, much bigger than that. But that data in transit, how does, where does SNAP logic fit in, in terms of being that connector, in terms of compliance and security and resilience there? Right, right, right, right.

26:20So Snap Logic platform and how it's actually utilized, we're very compliant with data governance and security, right. So in dated governance, there's RBAC, permissioning, auditing, full data metadata management right inside of security, there's an encryption API security, of course Nexus controls or SoC 2 compliant as well, right. OK. In a compliance space where you know, we're compliant with GDPR, HIPAA and of course, as I mentioned, SoC 2 compliance. So, and I'm not sure if I spoke about the architecture a little bit before, but essentially there's there's a separation between how snap logic's constructed, there's a separation between a control plane and the data plane. OK, you know, I didn't quiz you on that one. Tell us a little bit more about that then. So, Peter.

27:20Sure thing, sure thing. So if you think about the control plane as being like your management console of your system, right? And the data plane is essentially think of this as your execution engine. This is where the customer data would come to. And essentially you would use a control plane to set up your pipelines like I was talking about before. OK, OK. You. Use your data plane to essentially execute on those pipelines right? So and the data plane can reside. We offer what is known as a Cloudplex. Right.

27:56That that resides in the cloud. We, we can, we can, we can assign you one of those or if you do choose, but we also support what is known as a ground Plex, right? So you can actually download the software or if you want to run it in a docker, we also have a docker image as well that you can actually run a data plane either on your premise or in your own cloud or your own tenant, essentially. So we have both choices. And so in, in full compliance like none of the no customer data would ever come to the control plane essentially would stay, OK, that's your own premise, right? So yes, it's very unique architecture in that sense. OK, OK.

28:40Now that makes sense and it becomes very clear now because I would imagine that's probably one of the first things the client, new client might say is, you know, hold on, Especially as you mentioned earlier, people in the healthcare sector and obviously in the financial sector as well too. And, and all of those areas where you know, that it's key that, you know, data needs to have a ton of controls on it, very, you know, very, very sensitive data, customer data, the works. And, and I suppose look, Mongo DB faces that as well too. And I think we've ticked most of the boxes these days as well to make sure that usually when these queries come up from potential clients, they can be dismissed quite, quite easily by every, all the scrutiny that we've put ourselves to, as you mentioned, kind of the, the Hipac and the Sock 2 and all of those things that Snaplogic has assigned to as well. So we touched on, I get the mechanism again, look in the demo, we're going to see this really, I love the conversational piece of these live streams, but I really like the demo.

29:41So I'm looking forward to getting to that. But before we do that, tell us a little bit about, we've talked a lot about how Snap logic works and how it helps companies, etcetera. Tell us a little bit some of the useful practical use cases that you might have from from from customers, you know, can you talk to us a little bit about how they might be used and leveraged before we get into the demo portion? Yeah, sure. So let's say, if we think about like joint ventures, especially with both Mongo DB and Snap logic, I think it, it kind of plays into the new compliance space and also the multi cloud environments that we really want to get into, right?

30:21Because essentially that's where customers are starting to go to as well. There's actually like there, there's a Gartner report saying that actually by the end of this year, end of 2020, 590% of all large enterprises will be multi cloud or in environment as well. So it, it that's just phenomenal to hear about because because it's not just a fad, it's a directional shift in technology. So if you think about that space, you have to think about where data can then reside in multiple data stores and multiple clouds on premise, maybe even on my desktop back, my workstation that used to be under my desk years ago. So just to have that flexibility of being able to transfer data and transform data and also integrate with applications as well across all that easily, right? And officially is is like a mind boggling puzzle that that's always evolving as well, right.

31:26So I, I think that's probably one of the most ultimate use cases. I, I, I see. Hopefully that makes sense. I'm trying to. I'm making a lot of assumptions without going to. The yeah, no, no, it it definitely is. And I look, I think throughout our conversation, we've touched on a lot of it. You know, the, you spoke about the IoT and the edge computing earlier as well too. So collecting and, and processing that data nearby before putting it back into the resulting database or, or whatever the case might be as well too.

32:01And we talked a lot about compliance and data sovereignty as well. So I think that all makes sense. So I think if if it suits you, Peter, why don't we, you know, get into the meat of this and show a little bit of the snap logic UI in action and how straightforward that is, is it that suits? You yeah, sure thing, sure thing. So I have a a video to show after this but I wanted to just show this as a precursor. Sure what? What you're looking at right now is I'm logged into the stop logic control plane.

32:32Right. So this user interface that you essentially get. So if you look around this, it's a little you, I'll come to this stuff in the middle in a second. But essentially you'll see some of our functionalities and areas here. So you can see pipelines, right? OK. And then this takes a little bit longer just to load, but essentially you have different snaps. So these are different snap packs I was talking about before. OK, OK. The mongo DB snap pack you actually see all the different different that are available right? Perfect.

33:11Yeah. And here is essentially what is known as a pipeline, right? So there's essentially a start and an end, right? So if you look at this, it's it's like inside the configuration, you can see what it's trying to read. This is just a sample pipeline for for demo purposes. But essentially it's just you see it's reading a file, a CSV file doesn't need any account data. But if it was, you can input it here, OK. And then it goes to the next transformation step where it parses out and so on and so forth. And one of the end results is that it actually writes it out to a file called directory dot Jason, right? OK so if I was to execute this, basically it would read the file, transform it and then basically write it to two different different files.

33:59OK, OK. And usually I would execute this and then demo this, but I would just wanted to throw this concept out here because I wanted to show you something that's very exciting coming from Snap logic before I start this and and taking what I showed you before, essentially you can build an API driven pipeline. It's known as a trigger task, right? You can also front load that with, let's say you had a conversational user interface, right? And then let's say you built that and you wanted to integrate this into your ecosystem, right?

34:41A lot of the stuff you can actually get off the shelf in this one example I'm going to show you is, is using artificial intelligence. It's going to connect to an LLM. And basically an administrator can then purview and then use a conversational interface and search for items about iPhones, right? That's what all it is, and what it would require is interrogation of databases, so on and so forth. But no, there's not one line of SQL that's actually written. Sorry about the SQL.

35:13OK, we'll forgive you. But imagine if you will, it could also be among the database as well, right? So but the, but the concept here is that no one has written any programmatic query language, right, to view the, the databases, but they're able to you're, you're able to build this pipeline that uses AI to help provide the results that you are seeking. So I'm just going to let this play. This is going to go on for about 4 minutes. We're all showing the whole thing but, but here we go. Here we have a database query agent connected to a chat interface so that natural human language could be used. This gives business analysts and others greater and faster access to data by removing the need to create complex SQL expressions. Gone is the bottleneck of having a single data team responsible for servicing all data query request. Our database contains smartphone product information, so let's ask a question about device pricing for an iPhone. The AI agent processes our request and we get back a response in straightforward natural language. To build trust with AI solutions, we added the ability to look behind the scenes at the

36:31inner workings of the AI agent and show how it got the results. Here we see that the database agent leveraging another agent called Query Agent and passing it the parameter get the price for iPhone 13. That agent processes our original request and constructs ASQL query. SQL expressions are fairly specific and exact in nature. Things like spelling, capitalization, and spaces matter. In this case the product search was for iPhone 13 as one word, all lower case and without a space. Unfortunately, we see that the query returns no results. Instead of giving up, the database agent tried again but expands the search parameters and looks for any product with iPhone 13.

37:22This then gets us a modified SQL expression where the database product names are normalized to all lower case. Again, the query returns no results. Not to be defeated, the agent further expands the query once again, this time asking for all product where iPhone appears. The resulting SQL expression normalizes all product names to lower case and is looking for any product name with iPhone in it. This time the query is successful at retrieving information out of the database.

37:57In the next steps, the information is analyzed and then a response is crafted using natural language. Let's go further behind the scenes on the workings of the pipeline that makes this all happen. In the Snap Logic Designer, we have a main parent pipeline that defines the LLM prompts and controls the looping plus the overall flow of the agent. The planning agent takes the prompt from the parent and interfaces with the Amazon Bedrock LLM to call the query agent where the query is constructed and run. If no results are returned, the flow goes back to the parent agent for prompt refinement.

38:38You should notice that we haven't added any hard coded structured rules or logic. The pipeline is basically taking a prompt as input. Constructing ASQL query based on the prompt does the search and if nothing comes back, devises a new adjusted prompt for a new query. This dynamic creation of prompts, responses and evaluation of the response automatically refines the answer. Once data has been successfully retrieved, the next pass through the planning agent sends the information to the data analysis and response agent.

39:16There it is examined and a response will be crafted in natural language by leveraging Snap Logic Agent Creator. Create and deploy agents faster with any connector, pipeline or API. Avoid vendor lock in with your choice of LLMS and multimodal support. Safe and secure with built in observability evaluation and data security. Visit our website to learn more about SNAP Logic and Agent Creator. Excellent. No, thank you. Thank you for that. It was I, I understand what you mean now, but kind of we, we set the scene, we showed a bit of the UI and then we showed that in action, which which is great, you know, so, so. All of that, as you said, done with no code, with all of the snaps, etcetera.

40:05But I loved how you engendered the trust behind it because you could see what was happening behind each one. I think that's really key for building these types of agentic kind of applications is to be able to, yeah, we want, we know the AI is clever enough to do it, but we also want to look under the hood, right? We want to see what decisions have been made. Tell us a little bit about how that was kind of thought about internally and Snap Logic coming up with this, because I think it's probably key for people's kind of, I suppose, trusting of the agents in this instance, right? Right, and just look, with any type of new technologies and especially a a large evolutionary step, it's very important to put in checks and balances, right? Because there we AI, as fascinating as it can be, you still need to check where it is, right? Because obviously, right, currently now there's still the paranoia of like AI is going to take over the world and War of the Worlds is going to come and computer reverses, right, the whole thing.

41:13But, and so maybe it's checks and balance and a lot of that may be real, right? But it's certainly a valid concern. And just like any essentially AI driven pipeline, so to say, you'd want to put in the checks and balances as well, right? So you can actually put in pause steps in there where you would require manual intervention to inspect the results and then like continue another pipeline, right? OK. So, so essentially, yeah, that's a very important point and and and very a very valid concern, especially in today's day and age. OK, I love the way in the demo it refined the query a couple of times until it got, you know, it surfaced up some of the results that it was looking for.

42:02How do you like? Is there a limit to how you can, you know, would keep the agent would keep trying to refine a query to look at a result, or is there a way to put guardrails on the extent to which it might do something in that regard as well? Peter, is that a case? Yeah, exactly. It depends how you want to align up your snaps, right? Let's say counters in the snap saying, hey, make sure you know, if it comes back with no results more than three times, then let's take this other route.

42:35OK? So yeah, it's all configurable in there. OK. And obviously when you showed us the the the snap UI there and the snaps that were in there and this the Mongo DB snap packs that you had available as well too. And ultimately, you know, the whole goal of some of kind of getting the word out there about snap logic is hopefully to drive some attention towards that. How do people get started with snap logic, like how easy it is to, you know, sign up for the platform, join and and get to play around with that?

43:06Is there, you know, how do you get people on board the platform? Do you have amazing documentation, tutorials, etcetera? What way do you on board new customers? Yeah. So, so there's both, right. We, we actually have a wealth of documentation online. There's also there's an online community called Integration Nation as well. If you, if you did, if you did a search on Snap Logic Integration Nation, there's an online community as well. And you can also sign up for free trial, right? So there's that as well.

43:39And I think we'll share those links with the show notes, I would assume, right so. Yeah, yeah, yeah. Happy to put those out as well too, which which would be great. So people can sign up for the free trial. What is that? Is that a time limited trial or is it a certain amount of snaps or pipelines or what? What's the gauge on that, Peter? Yeah, exactly it is. It's a 30 day free trial, OK. And only limited to certain snap packs as well. I am curious if the AI ones are in there, the pipe loops and stuff like that are included in there. They may not have access to certain advanced features such as like API management, which we do have as well. That's OK.

44:22Very up and coming feature that we do have. I mean, it's been there already, but it's going to be a lot stronger in the in the coming months as well. I'm not sure that's included in the free trial, but I, I, I, yeah, I'm sure you hear about it. Unless. And in the the trial that they might do, they can still connect their real proper data wherever that might be up to their application or out to other providers as well too, right? There's no constraints on what they can do. It's it's just the time, right?

44:52Exactly, exactly. Yeah, yeah. So the the software is real, right? So you can, I'm not, I'm not, I'm trying to remember if you do get access to a ground Plex, which is basically downloading the software onto your own premise, but you would certainly have access to a cloud Plex, which is one that's managed by Snaplogic, so. OK. OK. Well, that makes sense. So just go to the Snaplogic website to click the trial and get in there. Is that the case? Yeah, there's a certain link. I think it's snaplog dot dot dot dot snaplogic.com free hyphen trial.

45:32OK, free hyphen trial. Perfect. I'll go to that URL right now while we're chatting, see if I can grab it and and drop it into our comments as well. I've only only 5 available. No, I'm just kidding. Not just maybe fine. Perfect, perfect. So I know we only had a short piece of period of time we saw the snap UI. For me, I thought when I saw the list of snaps down the left hand side that the fact that only the A's fitted on the first screen before the scroll.

46:02How many do you have? Like, do you know offhand, Peter, how many snaps snap logic has at this point in time? I'm trying to say I think it. I mean, I know it's well into the hundreds I'm seeing. I'm, I'm trying to think of it's surpassed 1000 mark or not. And then probably has and and there's more being added every day, right. So we also, not only do we develop additional snack packs, we also have partners that are developing additional snack packs as well. In addition to that, if you wanted to develop your own snap pack, we actually have a developers, you know, an SDK that would enable you to do that as well. So if you wanted to create a custom step, it's all there, right?

46:44Obviously that would require some coding, but. I found that URL, I just I just put it into the the comments there as well too. I'll post it across into into LinkedIn just shortly as well. At the same time, you can certainly jump there. I suppose the key here to also being the Mongo DB Podcast live. If you have your data on Mongo DB, you can use all of those snap packs and everything else that you have already populated up there to get playing around quite quickly, right? Yes, that's right.

47:20Excellent, excellent. What's excited you most about kind of this is the question I ask most guests because I I'm kind of keen, even from a personal perspective to see what they tell me. The AI you mentioned, it's been around for a long time, machine learning, everything else for those of us who've been around a lot, but you know, even in your age and creator video there, it seemed incredibly powerful and very, very straightforward to get up and running.

47:47What's exciting you in general, not, not just in your own snap logic space, but in general about AI these days, Peter? I I'm really curious about the next step AI is going to take the when, when we think about agents, right? Those are essentially it's supposed to be autonomous engines that run without any interactions, right? So if you wanted it to do something for you, use ask the agent, right? It's basically like if you can think of it as a music agent, right?

48:21Like what does a music agent do for you, right? So they help you book gigs, they help you market your your music, right, so on and so forth. So looking at the next evolutionary step of this is having everything be autonomous, right? That's pretty wildly fascinating what it could do, right? So, you know, can you, if you could just imagine, just like, you know, hey, go, you know, it's, it's, you know it, it's my, my wife's birthday coming up right. So like, oh, I really need to get her something and you know it's going to go and order it for me. And you know your.

49:01Wife's not going to like that example. Now, Peter, that you're going to use AI to buy her her birthday present. We have to think of a more mundane example, right? Yeah, exactly, exactly. I mean, I'll, I'll adjust, but yeah. Yeah, no, I agree. I mean, I for me, the agentic space is very interesting. I do hope it does take care of the mundane things to give us more time to do the nice things. Often, you know, all too often I see these amazing AI demos that they're doing the amazing cool stuff that you'd like to do yourself as opposed to the mundane stuff that you'd like taken away. But at the weekend I was away and I'd made a bucking in a restaurant that we couldn't make. We weren't going to get back on time. I phoned up and it's said, you know, the usual interface press 1, etcetera, etcetera.

49:48I did that and was like, and this is the first time in real life that happened to me. It was an AI voice agent. What do you want to do? You want to cancel your reservation Is that the reservation for tonight for four people at 8:30? Yes, it is perfect. Just confirm you want to cancel it it's cancelled and then done so for something like that, AI is perfect because that's a that's a non transaction. Basically it's like, you know, this is a this is a poor thing. I need to cancel it.

50:15I'm very sorry, I can't turn up, you know, whereas the flip side of that, I want to book a table. I don't know, do I want to talk to the AI per SE? I might want to talk to the the the host or Hostess and, and learn a little bit more about it. So we'll see. We'll see. Yeah, that's right. Just like the travel agent is a great example. Like if you're in flight, yes, let's say that it's going to be late, but you're going to miss your connection. You know, wouldn't it be beautiful if you had an autonomous agent actually rebook you on another flight while you're still flying on the first flight, by the time you touch down, it's seamless. Like, you know, because that's probably one of the most paranoid paranoia events that would happen when you touchdown like, Oh my gosh, my luggage.

51:05What do I do? How do I connect to to my other flight? How do I get to my destination? Well, if that's all taken care of for you, that just it just, it's just so much more convenient, right So. Yeah, I, I think so. And look, I very much look forward to that level of convenience hitting us all soon and various, various kind of projects etcetera as well too. Just before we wrap up, Peter, going back to, you know, at the beginning, very keen on your career path to date etcetera and where you've come from. I'm also keen in your role as as principal architect with Snap Logic.

51:39How do you learn yourself? How do you keep on top of the changes that are happening in our landscape all of the time? Where do you go to to keep on top of all of that news and technology? Well, well, I, I do try to stay connected with industry and different external partners like yourself, Shane, I mean, 'cause yeah, you bring a lot of great insight into and that helps feed ideas and like the next evolutions of designs and architectures that we do have in house for Snap Logic. So I think that's, that's a key opportune time, key opportune function is to stay connected with third parties, right? And and if if it was hard to get started there maybe going out to like different IT networking events in your area, if there is any or online forums as well, right? It was a lot easier pre COVID but but it's still possible today to do.

52:37That it was it's it's coming back and I know look, yeah, I think everybody experienced that we went through the collective isolation of being stuck at home. But I know from a Mongo DB perspective, we're, we're back on the road with all of our dot local events in many locations where I'm particularly involved on the developer relations team here at Mongo DB and running our the, the third party events that we're involved in. So these would be the developer focused events, the language community events, the the Java, the Python, the C# events that we, you know, have huge developer communities in and we turn up with those.

53:13So I agree with you, Peter. I think those sort of events I think I used, you know, there was there's a serendipity involved in turning up with these things and kind of having those hallway conversations and and meeting those people that you wouldn't necessarily kind of bump into. Because I think our inboxes are swamped with newsletters and emails that we like to keep up with. But I think it's often. Yeah, get out there and about is a good way I find that I learn, you know, you're you're away from the day job, you're paying pretty much 100% attention to whomever might be speaking on stage if you're at an event.

53:48I love that. So yeah, community events and yeah, paying attention to what's going on. It is super hard to keep up with how fast everything is moving though. Peter, right? It is, it is Frank, because today my car drives itself and yeah, I can order food instantaneously. So it's just, it's, it's, uh, the future is here, right? So. Exactly, exactly. Well, look, I think that's probably, uh, on those kind of words. It's a good way to wrap up our show. It's been great to have you on board, Peter. Um, I thought, you know, what we went through in terms of kind of where snap logic fits in, in this hybrid multi cloud world that we live in at the moment.

54:34I think the the ability to kind of understand the need for tooling like snap logic, coupled with the fact that you showcased the UI, which was to me the quick glance that I had of it, incredibly intuitive and obviously, as you said, more than 1000 or more snap. So I presume there's something for everybody there. Anybody who's trying to connect services together and not do, as you said originally was write a little bit of code that will do that, that will eventually break, right? Or that that, as so often is the case these days, that developer moves on to a different company and nobody knows how that code was put together, right?

55:11Exactly. And thanks for having me on, Shane. I really appreciate this. This has been wonderful. Not at all, no. It's it's been great to have you on board and it's great to hear that not only snap logic users of Mongo DB, but also connecting customers to their use cases of Mongo DB as well too. So it's nice to see the loop closed in that regard. For those that are joining us, we, we threw in the comments a link essentially they're to snaplogic.com/free Dash trial that you can go in there time bounded of course, as most trials are, but you can go in there fully function and get to play around with it yourself. I would be remiss of me if I didn't make it the odd plug for Mongo DB as well too.

55:54So everything that we do within Mongo DB, this developer focused either in the developer relations team or our engineering team and product team, we put up our developer tutorials on developer dot Mongo DB dot com. Please go there to have a, a good look around at what we have. We've got some great filters there. So you can find if it's a language, if it's a cloud partner, if it's an integration that you're looking for, by all means check that out as well too. And if you some people said they, I think in the comments, as I watch some people are just getting started with Mongo DB etcetera as well too.

56:27So if you need help, ourcommunity.mongodb.com is the place to go. That's our forums where our dev REL team, our engineers, our support people all hang out as well too. So that's the end of my plugs. But for me, Peter, it's been great to have you on board to learn a lot more about Snap logic. And it would be remiss of me not to do a shout out to your colleague Dominic. Dominic I met at Reinvent, AWS Reinvent. He's an ex Mongo DB person and essentially the chat with him at Reinvent going back to our talk about events being serendipity, quick chat with Dominic at the Mongo DB booth was, hey, can we get someone from Snap Logic on the Mongo DB podcast live? And here you are.

57:11So Peter, we're closing the loop. Dominic, gratitude to get Peter on the show with me. I appreciate that. I hope you enjoyed our slot Peter, and you got everything that you wanted to do across in our conversation. Yeah, it's been great, Shane. Thank you so much and thank you, Dominic. Yeah, thanks. We give him a shout out. Well, listen, Peter, it's been brilliant. I hope you know, I'll keep an eye on what's going on in Snap Logic. And as you have new things or new demos, just keep in touch. We'd love to get you back on the show sometime in the in the not too distant future to show how Snap Logic is, is kind of bringing new features to the this space as well too, which I think is incredibly important.

57:51As you said, it's a. It's a can be a messy space. It doesn't need to be with the use of tools such as snap logic. All right. Thank you, Shane. Will do, will do definitely. Excellent. Plus it's been my pleasure. Thank you so much, Peter, and for me, Shane McAllister. Please tune in every Tuesday mostly for you'll hear me or one of my colleagues have great guests such as Peter from Snap Logic on as well too. We do appreciate everybody who joined and commented and we look forward to having you back again.

58:19But for now, for this stream, for this episode of Podcast LIVE, thank you everybody. It's been great to have you take care. Thank you.