Name: Mastering Loops in IO: Handling Complex For & While Iterations Pt. 1
Uploaded: 2025-06-03T15:02:25.775Z
Duration: 30 min 12 s
Description: Mastering Loops in IO: Handling Complex For & While Iterations Pt. 1

Transcript for "Mastering Loops in IO: Handling Complex For & While Iterations Pt. 1": Hello, everyone. I'm Kelly Eiser, a senior solutions architect at Celigo. Thanks for joining our session, mastering looping in Integrator IO. Today, we're zoning in on one of the most essential yet sometimes misunderstood features of the platform, leaping looping through data. Whether you're a developer building complex integrations or a business analyst configuring workflows, understanding how Integrator IO loops through the data is instrumental on building successful flows. By the end of the webinar, you'll understand how Celigo loops through top level and nested records, when and how to use the one to many features, and common use cases for looping. So let's dive in. But before we jump in to the live demo, I have a few slides I wanted to share with you in order to set the stage for that live demo. So first, what is looping in Integrator IO? Looping within a JSON record, is basically when Integrator IO encounters a JSON, it intelligently iterates through the data it is, presented with. Looping means that for every item within the designated array, Integrator IO will perform a defined mapping based on the fields within that record. Typically, within a JSON object that's returned, there's two, levels that we're looking at. We're looking at the top level, which can also be called the parent, and then the nested records. The top level records, this is gonna refer to the primary object or set of data that defines that record that you're working with. For example, you know, the customer data or a sales order header data. If your JSON simply has one root object, that's gonna be your top level record. The nested records inside of that data, you know, these are gonna be the objects within that, top level. So a common example is a sales order containing multiple line items or an invoice with multiple, sales line items in there as well. So here's a quick example. So Integrator IO, when we pull a JSON object, it uses concept called the loop path in its mapping. The path tells the integration which part of the JSON structure it needs to iterate over. In the entire JSON example here, what you can see is we've got the top level and then the line items array inside is gonna represent your nested records that we're going to work with. So in the export step, when you first pull it out, the default behavior for Integrator IO is it's gonna begin processing data at the top level fields. So for in this example, it's gonna be your order ID, your customer name, the order date, and such. Then within that, you have your loop path that we're gonna talk about in a bit. For your loop path, that's gonna be line items here. In this case, there's three separate line items that it will that we'll wanna iterate over in in order to make this record go incorrectly. So, you know, if your import needs to create separate order lines, you know, you would set that loop path within Integrator IO to line items. Why is it why is looping powerful? Because it gives you efficiency, flexibility, and standardization. So, you know, understanding how Integrator IO is going to loop through those distinct levels and such is gonna be fundamental to building out your successful integrations. Alright. Let's start building our flows. So what I'm doing here today is I'm going to sort of start off by showing you how Integrator IO automatically loops through records returned by an export. For example, let's say that your export returns five records. We can configure the platform to automatically process each one through mapping and into any imports or such without having to write a looping feature or writing, like, a script or an if else statement or such. And I'll give you some examples of that here in just a moment, as well. So that's kinda where I'm gonna start. Then from there, we'll dig in and go through that. But first, to get started, let's start take a look at what we've got here. So what I'm gonna do is I'm gonna use Shopify and use their records as an example. So to get started, what I'm going to do is use a Shopify orders export. So what I'm gonna do is I'm just gonna grab an orders and then some particular fields. So what you can see here is I am using the HTTP method. So I've defined my connection, and now I'm gonna make a call to get some, orders. So if I select preview, what you're gonna see is that by default, when Shopify returns the records, they place them into an orders array. So right here, if I collapse this, you'll see I get multiple orders inside of that array. So if I then expand that, then in here, I can collapse that and see there are five orders here. So what I can do here is if we preview it, if we just leave it run that way let's see here. I've got everything configured. I'm just gonna take one of those records. I've got everything set here. You can see what I'm gonna preview here is if we just leave it set that way and work with it, if I go into a next step. So if I just left the call as is, went to the next step. Let's say I wanna do a transformation or hook or such. But if I start to work with my records here, you see that the input data over here, I have to work with this array. So it's orders, and then I have to figure out how to loop through each of those, accordingly. So I have to set it and get everything configured. Same thing here with my, import. So here, what I'm doing is, I'm just gonna write out to a little database that I have that I can work with. So I can go in. You'll see my import. It's just a post call. Nothing fancy there. And then for my mappings, which you'll see here for mapping, is I'm just mapping into my field. So here in my, database, I've got a field called data from time. There, I'm just gonna put the actual record, and then I've got a couple fields here, input data notes, which are text fields. And that's what I'm gonna populate there. So, what we can do here is now if you see if you were to look at mappings, but here in my mapping, what you can see is I just have some data being populated. Because it has the orders array over here, to work with that info, like, let's say I wanted to have order number or such populated, you see here that if I select that, I have to deal with the array somehow. Right? Makes it a little complicated and such, but in this case, I'm just gonna take this out. But I want you to see that if I remove that right now, in in test mode, what you'll see is if I run test mode, Integrator IO is actually going to sort of make those calls and process through that data without actually posting it to my endpoint here. So you see that was still empty. And if I wanna preview that data in test mode, what I can do is actually open up each step. And if you'll see if you're in the top right, what I can do is open up my, debug log, look at the test debug, and it'll show me what I just performed, what action there. So you could see there would be, the request, with some context. So, like, there's the actual API that was called. Here's the response that came back. So there's the raw response. You can see here all my records are within that, array of what was returned and such. So there, I can see that. Then what I can also see is if I go to my next step here, if I open that up, I'm able to look at the debug log. Now this is where you can see if I just leave it as is and run through this, when I actually post this information into my endpoint, it's going to post that in with a one single record here. So it looks at the request. Give me just a second there. So you'll see here at the request, it's gonna post in that JSON. So, basically, our our mapper is going to build that request. I've got one. And then here in the JSON data, you can see I have orders with everything inside of one record that I have to deal with. Right? So you can view that and test it. Now I'm actually gonna enable this here and run it because I want you to see how this is going to post to that endpoint, actually running it in this manner. So again, you know, we're we're trying to explain here how Integrator handles iterating through data. So when do you use a load, when you need to use a more, you know, in-depth solution. So I'm actually gonna run this now. And here you'll see in my run log, you see it pulled one it says one record. Well, that's one page. So that's gonna be that order. So you could see it completed, says it posted out. So now if I go and look at my data, you can see over here, it posted this, and I have a single result. Okay? But now let me just expand this here, and you can see same concept in that orders in the JSON data. If I look at the way it handled my data, it took that single record because Celigo Integrator IO handles that page, the orders object, and it gave me all five records inside of an array. So now I, you know, would have to deal with that somehow in some way if I wanted to, you know, iterate through those and get those out separately. So let's clear this out so that on subsequent runs, you can see each of those. Now what I'm gonna show is, just with a couple changes within my flow, I can have Integrator.io automatically, like, loop through those records, without having to write any additional logic or anything. So to do that, what I can do is if I go back to my export, and we'll hit preview here. And, again, you see right now when it comes out, it's in that orders. But if I go down here and I expand the section called nonstandard API response patterns, what I wanna do here is it's asking where what array do I wanna go to? Where's the path to the records I want to work with? So if I go here and put orders and preview that, you'll see that now it doesn't have the orders object there anymore. It has the individual records. So I'm already now inside of those records. So I don't have to do any extra step to loop through the returns. I wanna expand this because I wanna go through five records now. I don't want one array of records. I now wanna go through those five individual records is what I'm gonna do here. So now you can see I just made that change. I just expanded the orders. And now if I do save and close now before when I opened up and kinda showed the example on this transformation, you saw I had the orders that I had to work with. Now here, you can see I'm already inside of one of those individual records. So, you can see the information, like order number and such. So I can, you know, now start to work directly with that field level data. Now what I'm gonna show is how that changes the downflow results and such that I can work with here. So now, in my next step, I don't have to make any changes here. I'm gonna leave everything set the same as is. But now if I go to my mappings, now you'll see I'm already inside of that record. So, you know, it's very easy to work with that data for each record. So for example now, I'll just populate this in my field and put data in my database. And now here, you'll see I just click inside of here, and you can see it's showing me all my fields from my individual record. I don't have to deal with that orders array. Yeah. The tool's already gonna loop through all the individual records out. So, if I hit save and close now if I run it so where before I pulled out one array and it put one array record inside of my database there. Now if I run this, just with those, couple of changes, you'll see now it's showing that it received five records. So it knew it already went through and looped through those five individual records and took those and posted. So now if I go look at my export results let's refresh this. Now you see on my export, Integrator IO already looped through each of those records. So now you can see here's my order numbers. You can see it went through each of those records and then passed it in so that each one could be handled, individually. So now if I were to open one of these and look, you can see here is my order, you know, information, tax line info, and each one is now in there accordingly. There's all the line items, and you can work with that accordingly. So, again, you know, very easy to see how with just a couple changes, Integrator IO, is automatically going to loop through the those records for you right at that state. Now, again, let's go ahead and clear out these records before we run this again so it's easy to see on our next run how Integrator IO is handling those records. Next, let's now look at each of those individual records. So we now walk through how to take individual records and loop through those from the response. Now let's consider that each of those records coming out as a parent record could have a collection of data inside of that, so another nested array. You know, an example is, like in this case, you've got an order with multiple line items. You could be doing projects with multiple tasks or, you know, invoices of multiple line items and such. So for example here, let's look at this order example. If we look at that export now so, again, we still have it processing each individual order. But now inside of this order, we have an array for tax lines. So you can see the array designated by the brackets. Now our customers is just an extended object, so it's not an array. But then you've also got your array of line items. So what we enabled or what, process we have here within Integrator IO is you can now enable the one to many functionality on an import step. So, essentially, what that is going to do is if we go to our import, you can see here that we now have this option says one to many. So if I select yes, you'll see here it's gonna say you it's gonna ask you for your path to the array you wanted to iterate over. So for example here, if I select the drop down, then you can now see there's my tax lines array, there's my line items array. So in this case, if I wanted this to iterate over each of those line item objects, if I select that, what that's going to do is it tells the step to go ahead and, go within that array, within that object, and we'll iterate through that. So just by enabling that and selecting my path, if I select save and close, that's all I have to do there. Now you're gonna see if I go into the mappings, if I look at my source data here, you're gonna see how it changes. So now, you see here how we now have underscore parent. So that parent level data is still available here. Right? So order numb your original order number, the contact information. But if I collapse that, you're gonna see I've got direct access to each of the line items that are within the order there. So I can now work with all the information here. So now in my mapping, if I wanted to, I could get my order number. Now you see before when we mapped, when we were just looking at that single record, I had to go straight to order number. Now that I'm in the array, I actually have to go to parent dot array and or parent dot order number. So here, you can see our tool in the drop down. It'll even show you that automatically. So I can just go here, select parent order number. So I still have access to that parent level data there that I can work with. But if I wanted to now, I could add an additional field. So let me go here, in my database there. Little example. I think I've got I think it's called notes. Let me double check. Yep. Notes. So I'm gonna populate some data here. Now what I can do is in my source, and this is just gonna show you, you know, for example, that I I am working that that particular line item information there. So what I'm gonna do is I'm just gonna go in here, right, just a, a little expression here. So watch if I work with our editor. I'll open that up. So now here, let's say that I'm gonna collapse parent. I wanted to say SKU equals, and then I can do the two left handle handlebars there. And if I select out of my record, I can go through. And you see I'm at that line level. So I can just go through here, select the field I want. So I mean, I want skew comma quantity equals, and then same thing. I can type ahead and you can see it'll filter for me. So there if I do Q quantity and then maybe I'll blind you know, the dollar amount in here. So I'll do dollar equals record dot, and it's a price. So I'm gonna select just the top of a price. So I'm actually working with that record there. So if I save and close that, I now have a little, statement here that'll let me populate those three fields. So if we save and close this now if I run this, what you're gonna see is that for each line item on a record, it's going to put an entry in our database. So let's let this run. Now you can see in my output, it shows that we process seven records instead of five like we did before. So now if I go back, if I refresh my input, you can see here I now have entries for each line item in my table. So you can see that what it did is it actually handled each line item on an order and entered it. So here you can see the order, $15.99. It It actually had three line items. And then here's the input. There's my SKU, quantity, and the amount for each line item there. SKU, quantity, amount, etcetera. So that's how, you know, again, without having to do any, you know, JavaScripting and advanced logic, you can use the features that are directly within Integrator IO to have it loop through different array sets. Now that we have that that sort of concept set, you know, in future webinars, we're we're gonna delve a little bit deeper and then start talking about the, the way that Integrator IO actually will handle those datasets as it passes through each, different step here within the platform. So, you know, when you do go into a a flow and you click the three dots here, these steps are in order of operations. So depending on how you handle the data through each of those, it depends on how that gets applied. And in future webinars, we're gonna go through, you know, this looping and then how that data actually gets passed into, as you can see, like an incoming filter at the inputs. When we pass it into hooks, how hooks can use that data and such. So, I hope this was helpful. Thank you very much, and please let us know if you have any questions. Hello, everyone. Hopefully, that was helpful. You know, again, we're just starting this. We wanted to set some basics of, you know, looping how Integrator IO starts to work through the record. So if you have any questions, you know, please let us know. Glad to answer those. You know, otherwise, again, we're gonna work on future sessions to start digging deeper into how Integrator IO goes through looping and how that can help you. So, at this time, let us know if you have any questions. Austria, let's see here. What did you ask there? Or how how about if I need a field to be part of the array while looping? Do you mean from a parent that's in the parent record? You know, if that's the case, you could still reference it with that underscore parent, and you should be able to access it, you know, from there. That's where in that last example where I had the underscore parent to get to the order ID, you can also do that same thing when you're inside the loop. So it would still reference your parent field that you could include for each record that you're processing in the loop. So for one, should I have there? Let's see. So for one to many, as an alternative, can I use the create destination rows? Do you mean you want, like, three individual rows that I was creating there? If you're populating to a spreadsheet or database like I did there. So if you wanted to yeah. That's, I think my example there when I was posting with that database, it did create three, destination rows there. You know, the other use case for that one to many is, like, you're creating a sales order and say, like, your ERP where you create your header info, like a sales order number, the customer, and then those line items would create three line items on that order in the ERP. John, I'm I'm reading your message here. Let's see. Still been clear when you need to use the one to many option, advanced, and then import. Is it anytime you have a header, lines, objects such as a sales order, only if you're dealing with multiple records? Now the one to many, you know, that's where like I was just saying, yes. Sales order is a good example where you have a sales order that'll have multiple lines. So, you know, in your destination, you wanna have one sales order record, but you wanted to have, say, all three line items included on the one sales order. Where if you didn't use the one to many, what Integrator I would do is you would it would create three separate sales orders all with one single line item on it. Right? So you want one sales order with your header info, the order ID, the customer info. And then you want those three that you wanna loop through the line items within that JSON object so that it creates all those on that one sales order. So you'd use it there. So sales order is a good example. Invoices is a good example. If you're doing, like, receiving, like, item receipts against purchase orders, creating item fulfillment records against sales orders, you know, journal entries are are a good case where you have a single journal entry linked by an order ID. Like, again, some header information, like a journal entry order or journal journal entry number, but then you have all the line items that you wanna add to one single journal entry. So that's typically where you would use that one to mini. Alright. Any any other questions? Yep. Ankit, you know, I you know, interested to see the complex logic in the different hooks and scripts. You know, we're gonna build to that. You know, we're trying to build some foundational. This is these are the basics. This is how, you know, IO handles the records and breaks them apart. We're gonna do future sessions where we start to get into some more, complex use cases, you know, like how IO passes it through to different steps, you know, the pre hook maps, post hook maps, the filtering, and such. So, yes. Thank you for that input. We are going to be building into that. Alright. If there's no additional questions, you know, thank you for, attending. We appreciate it, and I hope everyone has a good, rest of your day. Thank you very much. Hello, buddy. I'm pop back in here. I see you had another question. Doesn't normal mapping handle arrays fine so you don't need the one to many? In some cases, it does. You know? Again, that's where it's kinda like math. There are multiple ways that you can handle arrays. In this use case, the one to many definitely makes it very easy just to make sure it's iterating over without having to understand, you know, the asterisk notion for your line items and such. So, you know, in in today's session, we wanted to highlight how you can use the one to many to explicitly tell to iterate through those. Alright. Thank you, everybody. Appreciate your time.