Episode 06

Headshot of Matt Shull

with Matt Shull of Aristotle Interactive

Our guest this week is Matt Shull. Matt works at Aristotle Interactive where he manages projects and helps shape performance best practices. He also teaches others about web design/development through Thinkful in his spare time.

In this episode, we talk about Matt’s experience with making sites fast for big clients, the tools they use at Aristotle Interactive to monitor performance metrics for their clients, the WebP image format, and a few performance pitfalls and how you can avoid them.

Direct Download

Transcript:

Katie Kovalcin:

Welcome. You’re listening to episode six of the Path to Performance, a podcast for everyone dedicated to making the web faster. I am your host, Katie Kovalcin.

Tim Kadlec:

And I am the other host, Tim Kadlec.

Katie:

So Tim, how have you been? Haven’t seen you in a couple of weeks.

Tim:

Yeah, it’s been a little while, right? We did the Velocity thing and then… Just kinda chillin’, enjoying that warm sunny Wisconsin weather.

Katie:

Playing Nintendo, I hear.

Tim:

Playing Nintendo, yeah. [laughs] I think that was more your thing. I thought—how many hours did you spend playing Donkey Kong?

Katie:

[laughs] I’ve been playing a lot on the Wii Virtual Console. I really want a Wii U! Is that, like, not cool anymore? Are they still cool? I don’t know. I want one.

Tim:

I would probably be the worst person to ask. I like the idea of video gaming and I used to do it quite a bit…

Katie:

[laughs] “I like the concept…”

Tim:

I don’t do it so much anymore. Yeah. But no, actually, you know what—

Katie:

You have kids! Don’t you play with them?

Tim:

Well not, like, video games…

Katie:

Well, you buy the video for them and then you play it with them and then you can end up playing them, right?

Tim:

Again, I like this concept. I haven’t actually executed it yet, though. But it’s a good idea. And I actually think that—I feel like that should be something… because you can learn from the games and how their animations happen and stuff.

Katie:

[laughs] Yeah, yeah…

Tim:

Oh no, no, this is legit—somebody was saying this on Twitter, or some sort of a chat somewhere, I don’t remember where this came up—but somebody was telling me… it was talking about perceived performance, and they were talking about how much perceived performance comes into play in game design and how they used to build their animations of the characters so that they could have the animations take as long as needed to cover up for the latency. But at the same time, the animations could also jut out really quickly if things came back quickly. So like, they had perceived performance designed into the animations and stuff and that, by the way, should be our next episode.

Katie:

Yeah! Who do we know that makes video games that can come on the podcast?

Tim:

I don’t know. I can figure out who I was talking to and have that person talk to us.

Katie:

That is like a thing, because I got really into playing Super Smash Bros Melee in high school, and that was the one on Gamecube that was—it has a really good speed and people really like it because of the pace that everything is animated. In the N64 version, which a lot of people also love, but I hate it because I got so used to playing at the animation speed of Melee, and I’m just like oh my god, 64 is so laggy, I hate playing that. And then it was weird because Brawl, which is the one for Wii, is like a slower animation than the Gamecube one, Melee, but I think that they just changed the physics of it to be different, it’s not that the perceived performance is different. I don’t know, there’s just different speeds for, like, the same game and you have your preference… Moral of the story: Play Melee.

Tim:

Okay, so yeah, there’s two takeaways here. First is that we have to do an episode on this, perceived performance in game design, because that would be just darn fun. And the second thing is that, yes, I think that we have both basically just made a business case for why everybody listening should be able to go to their boss and justify a purchase of a gaming system and, I don’t know, what do you think, like five, six different games that they should be able to play on company time as research.

Katie:

Yes, agree—well, I don’t know, that’s like… I don’t know, that’s getting into like, “We’re such a cool startup because we have a kegerator and an Xbox!” kind of thing. [laughs]

Tim:

What do you have against kegerators? [laughs]

Katie:

Eh, I don’t know… Eh…

Tim:

Alright. No, that’s fair. That’s fair.

Katie:

You know, the, like, really typical ’brogrammer”: “We love bacon and kegs and video games!”

Tim:

And now you’re insulting bacon…

Katie:

[laughs] I don’t think bacon’s that cool!

Tim:

No… We went from a new high for the podcast, I felt, with the gaming design stuff to now a new low where we’ve insulted kegerators and bacon.

Katie:

[laughs]

Tim:

I mean, I’m with you on the brogrammer thing. But bacon, Katie? Bacon… That’s legit good.

Katie:

It’s fine. I’m not going to say no to it, but I don’t, like, lose my shit like it’s the best thing ever… Yo, let me tell you about something I did lose my shit over though. I was just in Vegas last week with a group of me and my besties, went to Gordon Ramsay Steak and we split this one steak that we didn’t know was going to be stupid expensive, we were just like, “Yolo Vegas!” and we got it, and it was, like, some really fancy kobe steak. And they cut it into fours so each of us could have literally a 1 oz bite, and it was the best bite of meat I have ever had! I wish I had more than one bite of it… [laughs]

Tim:

[laughs] The best 1 oz of steak ever.

Katie:

Yeah, Yesenia almost cried. She was like, “It’s so good…”

Tim:

That’s awesome. Did Gordon Ramsay come out and yell at you guys while you were eating?

Katie:

Okay, my feelings on Gordon Ramsay: I know that he’s, like, supposedly a monster, I don’t know, but I’ve only ever watched Masterchef Junior and he’s so sweet to those kids! I’m like, “Aw, Gordon Ramsay, he’s such a cute, nice, little chef guy.” Like, I don’t see any of the shows where he’s an awful human. So in my head, I’m like, “Oh, that’s the guy with the kids!”

Tim:

Yeah, I actually don’t watch much of Gordon Ramsay and I don’t know anything. He could be a very sweet dude. I know that I would be happy to have him cook for me, though. That I do know. Even if it was only a 1 oz steak.

Katie:

Yeah, that was good. Would recommend. I would leave a positive Yelp review of them.

Tim:

[laughs] Alright, just recommend having a burger before or after as well so you get full.

Katie:

Well no, I got other food too. That was our appetizer. [laughs]

Tim:

Oh, okay! I was picturing this 4 oz steak coming out as the main course.

Katie:

So, they bring a meat cart around…

Tim:

Ooh, I like this already.

Katie:

They wheel this cart of meat that has all their different steaks around it; we got a picture with it, it’s so good. But they’ll show like, “This is this cut of steak,” and tell you about the different ones. And so then like at the end of their little spiel about their meat cart, they pulled out this one and they were like, “But this steak…” and he was like, “Oh yeah, it’s all this stuff…” and we just totally fell for it. We were like, “Uh, yeah, yeah, we’ll try that,” because he said, “You know, I can bring it out and cut it into fours so you can all try it.” So, we were like, “Okay, yeah.” So that was our steak appetizer for the steak dinner that we then had. [laughs]

Tim:

Steak appetizers for the steak dinner. That’s perfect.

Katie:

Yeah, I was a vegan until a couple of years ago. So, that’s where I’m at in life.

Tim:

[laughs] Oh, and see what you would’ve missed out on?

Katie:

I know…

Tim:

No, I’m with you. That’s exactly what you have to do though. How many times do you go to a Gordon Ramsay restaurant where you can have that sort of—like with a meat cart and the steaks… You have to do it.

Katie:

[laughs] I also made them wheel back the meat cart before we left so we could get a picture with it.

Tim:

I did see that. [laughs]

Katie:

Because we were signing our checks, ready to go, and I was like, “You never brought it back when I asked you to…” and he was like, “Oh yeah, I forgot.” I said, “Well, could you do it?”

Tim:

Did he legitimately forget?

Katie:

I don’t believe it. I think he just didn’t want to wheel the cart back to us.

Tim:

“This is not that kind of establishment.” [laughs]

Katie:

Yeah, “You’re coming into Gordon’s house, treating it like Applebee’s right now and it’s embarrassing.”

Tim:

Awesome.

Katie:

Back to, uh…

Tim:

Yeah, we started off tying the first topic into performance. I’m not actually sure how we got steak there. But that’s cool.

Katie:

Well brogrammers led into bacon not being cool, and then like, I don’t know, I was just trying to redeem myself by talking about meat basically. [laughs]

Tim:

You did, you did. [laughs] That’s good. I think you pulled it off well. So yeah, performance. Did you see the resource timing article that was out recently?

Katie:

I did see it! I glanced through it; it had a lot of—there were a lot of really good tools listed in it of things like, “This helps you in this way…”

Tim:

Exactly, yeah. There were some great tools down there. My favorite I think is from Mark Zeman, this heat map thing. Did you see the perf map? It shows you in colors how each block of your site loads and which loaded first and last, and it’s super neat. Yeah, the whole article is really good. It’s by Nic Jansma, and I guess he’s at SOASTA, and it is a really, really detailed look at resource timing—what you can use it for, how to get good metrics out of it.

And then at the end, he just walks through a ton of different open source tooling available, including Andy Davies, who we’ve mentioned before, his waterfall thing; the heat map thing. There’s a lot of cool stuff in there and some of it’s very visual too, which is nice vs. the typical, traditional kind of ugly tooling that sometimes gets put out. There’s some well-designed tooling in that list too, which is cool.

Katie:

We should get into well-designed performance tooling…

Tim:

We should. [laughs] Well you should. If I designed it, it would be Times New Roman text on a white background.

Katie:

I saw What Does My Site Cost, you put some thought into that!

Tim:

A little bit of thought. It was hard though. I had to pick colors and stuff… Anyway, yes, you should get into that.

Katie:

So there’s that video that you sent me that’s pretty cool. ServiceWorker is something I just learned about it and it sounds awesome.

Tim:

Yes, Jake Archibald’s video. It’s an awesome video. It’s just really well done. It’s one of those things where, yeah, he explains it well with examples all the way through. He basically takes an app and starts with it rendered entirely on the client, “now let’s move to server rendering, this is the improvement. Now let’s add Service Worker…” Like he just, step by step, walks through and shows you how to do it and what the improvement was, and by the end he has a super fast web app. And despite the fact that service workers can maybe be a little bit of an intimidating topic, the whole video’s done in a way that’s entertaining and hilarious and easy to understand and full of pictures of zoo animals.

Katie:

I like animals.

Tim:

Animals are good. So yeah, so definitely anybody who’s looking to get into service worker stuff… I mean Jake is probably the person pushing them the hardest, but that video is fantastic. Super fun. I’m jealous; I wish I was entertaining like that.

… Thank you for the silence. You’re supposed to jump in there!

Katie:

[laughs]

Tim:

No, that’s where you, as the other co-host, is supposed to be like, “No Tim, that’s fine. You’re actually very entertaining,” and then I, like, do the humble shrug off. “Oh, well thank you for mentioning that, Katie.”

Katie:

[laughs] I know, I was sitting here like, “Should I walk into this? Should I comment more on the video? Should I…”

Tim:

Wow. Yeah. Uh huh. Alright, let’s move along then…

Katie:

I didn’t get to watch both of your presentations—

Tim:

Don’t do it now.

Katie:

But I saw one and you were entertaining…

Tim:

No, you can’t do it now.

Katie:

BUT! You missed my, um, whatever the slide improv thing was and I failed to be entertaining there, so.

Tim:

Yeah, the karaoke presentation that you gave. That’s alright, I did a karaoke presentation one time and I was abysmal. The only thing that saved it was that it was late into the party when everybody was doing it, so everybody watching was well… intoxicated at the moment, so it saved me from being a complete and utter failure. But, you know, at least I didn’t get booed off the stage, nothing like that, but. Anyway, performance.

Katie:

Yeah, back to our podcast. Back to our regular spiel.

Tim:

Yeah, so those are both—like the resource timing article is excellent, the video is excellent, definitely check those out. But I think we’re going to kind of keep the intro pretty short this week because we have a pretty lengthy interview.

Katie:

Yeah, and it’s full of really good stuff. So yeah, let’s go hear a little bit about some agency performance culture.

Tim:

Sounds good.

Katie:

And we’re back with Matt Shull from Aristotle Interactive. Matt actually reached out to us by sending us an email with a really cool story about how he was able to get performance integrated into his organization, so we’re going to let Matt sort of tell his story. Welcome, Matt.

Matt Shull:

Yeah, thank you guys for having me. Yeah, so I came to Aristotle back in December of 2014. I’ve been doing a lot of freelance stuff up until that point through the several years, and one of the things that I’ve just kind of really been diving into is web performance over the last few years, and how to make websites fast and how to make that user experience phenomenal for the users.

So one of the things that, coming into this job, that I was really harping on and really excited about was bringing web performance to Aristotle Interactive. And they had been doing a lot of page load time metrics but using just that metric to measure performance. And so when I came in, one of the things that I started kind of talking to people about was web performance and some more meaningful metrics, such as time to first byte, Speed Index, and page weight. Those three metrics are kind of the three metrics that we’ve based our performance budgets on.

So coming in, there wasn’t really a performance team, it was just kind of an afterthought more of in the QA process after we’ve gone through months of creating the website, and the wireframes, and the mockups, and developing these incredible websites. And some of our clients are actually some very large clients and a lot of tourism websites, so like Gulf Shores or Arkansas.com, and then even the Elvis Presley Graceland.com is one of our clients as well. So, they’re websites that get a lot of traffic and they’re websites that a lot of people—it’s pretty demanding, they want information as fast as possible.

So, we just started looking at some of the performance metrics, like we talked about time to first byte and Speed Index and page weight for these websites, and we noticed that it wasn’t up to par as it should be, and so we just kind of threw together a team. This is something that I started heading up at Aristotle Interactive, and they just kind of let me run with it. So through this team, we’ve been going through and taking a look at our existing websites and saying, “How can we make these metrics better for these existing customers?” But then also with these new projects coming in each month, “How do we, up front, make sure that we set these performance budgets?” and we make sure that they’re built from the ground up being performant.

Katie:

So you mentioned in your email about things that you started doing with that team, having them sort of review best practices and teaching them about those best practices, and including them in design reviews and stuff. Can you talk a little bit about what that whole process looks like?

Matt:

Yeah, it kind of looks different for different people. Like whenever I started talking to different people about performance, there were some people that were just like, “Oh man, this is how it should be. We should be doing this from the beginning rather than at the end,” and so they were very excited, and so they were like, “Give me every blog that you can find on web performance. I want to read all of the information and all the books that you can find.” And Lara Hogan’s book on Designing for Performance—a few months ago I got done reading that and so I’ve been passing that out to people.

But then there were some people that weren’t so up for it, some people that were like, “Well, we’ve done it this way all these years and this is what’s been working. Why deviate from that?” And for those people, it ended up that we needed to show them data to say, “Look, this does make a difference.” And so, different people require different ways of teaching them why performance matters, but eventually they all come to the same conclusion, that yes, this is important and that this is something that we should be more attentive to.

Tim:

I’m curious, you were mentioning the three metrics that you chose; I’m curious how you go to those metrics. Why did you choose those metrics? And did you choose amounts for those that you were going to target across all your projects or was it per client? I guess, how did you arrive at those specific metric types?

Matt:

Yeah, so we’re in the process right now of determining—in the discovery phase of each project that comes in, we want to set certain goals, whether that be SEO goals or different goals like traffic goals, but we also want to do performance goals. And when we first started, we were like, okay: brochure-type sites that are just informational, the time to first byte should be a grade A for whatever WebPageTest.Org shows, and then also Speed Index should be 1,000, give or take a couple hundred, and then also page weight should be no larger than 1MB. For our tourism websites, we did a 1,500 page weight and 1,500 Speed Index.

Tim:

And why the difference?

Matt:

So the difference is the majority of the time the tourism websites just had more content on their pages; there were different things. Like one customer we’re working with right now, they want a video background—instead of this big hero header image, they want a video that plays automatically like you see on some sites today. And so there’s different things—tourism websites want to be more flashy, they want to have more content, and especially with different promotionals on that main page they just have a lot more content. So that’s why we gave them more of a budget than others.

And this kind of worked for a little bit, but we started noticing that it wasn’t probably the best way to go about it. And even with existing websites, we tried to set those same standards and for a few of our websites they were way off, and so we were like, “Oh man, what’s going on?” And of course we’d see that the time to first byte was high on one of ours, so we thought, well, there must be something server-side or the connection-side that’s causing it to be like that. So, our existing sites, we’re not necessarily able to just drop everything and fix all these existing sites and so it’s a kind of ongoing basis.

So for a developer or a designer, whenever you see that and you see that you’re not hitting the metric even though you’ve made all these improvements and all these changes, it can be a little bit disheartening. And so recently, just a couple of weeks ago, we were in the performance meeting and what we decided to do was the 20% rule, which is talked a lot about. So for existing people, existing clients, what we do is take their existing website and say look at the time to first byte, Speed Index, page weight, and we’re going to set a goal of reducing it by 20%. And then, if needed after reaching that goal, we’ll do another 20% until we get to where we feel it needs to be for that particular client. It helps a lot more with morale too, because the developers and the designers get these smaller goals that they can hit and then they’re like, “Okay yeah, this is working, this is going great.”

So that’s kind of been our process as far as determining what the performance budget should be. Like I said, we’ve tried to find more meaningful metrics for these websites in the discovery phase. So something like time to first tweet like Twitter does or something like that—we just haven’t necessarily found those yet. But we’re still exploring that.

Katie:

So you have performance meetings. What are those like?

Matt:

Yeah, those are interesting because it brings together people from different departments. One of the things I wanted to make sure of whenever we started a performance team: it wasn’t just people that were fanatic about web performance, I wanted people that were probably going to be skeptical of each thing because I wanted to make sure that we did the right thing to make a website more performant. So we brought together people from the design department, the development department, so a designer and a programmer. We have a hosting service as well, so we actually host websites, we design them, we develop them, and we’re an internet service provider as well. So we’ve got people from all different departments that come together and create this team, and it’s been really cool to see people latch on to this idea of performance, whereas when we first started they were skeptical and just like, “Do we even really need to be having this team meeting? Like, this is just another meeting.” But now they’re very involved.

And so what we do is we have a tool that I developed that measures real user data and synthetic data, and we have this big dashboard and we take a look at each of our sites and we say, “Here’s the sites that we worked on over the past week. Here’s the performance wins that we have,” and we love to celebrate those wins with our developers and our designers, and then we take a look at maybe the sites that did not go well or that all of a sudden spiked up, and our system is able to tell us “This website has spiked, this Speed Index spiked by 1,000 over the past week,” so we can take a look at that and see what went wrong. Usually most of the time what it is is the client—in an image rotator—they’ll upload this huge 3MB image for some reason. And so we’ll then call the client and say, “Hey, we noticed this jump in your metrics. We’d like to help you fix that.” So, it’s really amazing. I’m very much enjoying working with the team to work on existing clients and new clients.

Katie:

Do the clients use this tool as well or is just mostly internal for you to monitor it?

Matt:

Yeah, it’s just internal for us to monitor it. We have a service where they can use this service and have us monitor these metrics, and then every month they get a monthly report of the performance wins, things that got better. Then also we use other tools such as New Relic, to monitor the back-end performance as well and take a look and see what errors are happening, and we suggest maybe some things that we should be working on over the next month or so.

Tim:

So these reports that you give them, do they say, “Your Speed Index went down 200 points,” or, “Your time to first byte went down 100 milliseconds” or whatever it is? Or are you tying that somehow to their business analytics for their site in any way?

Matt:

Yeah, we are. One of the things that we’re using is another software to kind of bring together those business analytics and the SEO analytics, and then we take the information from the performance dashboard and meld those together to bring the report. And another thing that I really like to do is to show them the graph of the Speed Index just dropping like a rock. That’s one of the coolest things, is to be able to show them that. And they respond very well to that, because they’re like, “I am noticing my site be a little bit faster,” and yeah, they definitely appreciate the graphs and being able to spell out what it is that was fixed.

It’s funny because customers care so much about performance—they know why, they just don’t know what to care about. And so we love educating the client. One of the things is telling them total page load time doesn’t necessarily tell us anything, and so we tell them why Speed Index is important and why page weight is important, and even show them tools like… WebPageTest recently added that tool to see how much it costs to view that page on a mobile device, and so that type of analytics they just really respond well to.

Tim:

Very cool. Yeah, I happen to like that tool for completely unbiased reasons…

Katie:

[laughs] Yeah, Tim knows a thing or two about that tool.

Matt:

[laughs] I know, yeah.

Tim:

Very cool. So these tools you’re using then that are doing the business analytics and stuff, is this built by yourself as well? These are tools that you’ve built internally for everything? With the except of, like, New Relic, for example. But the other tools that are pairing your performance metrics and business metrics, are those custom built tools that you have in-house?

Matt:

The reporting and bringing together of the different information is done through Tableau, and it’s something that I’m not very well-versed on, but it’s something that our marketing team know really well and so I just hand them the performance data that has dates and different things. And then we’ll also, in the dates, we’ll include notes to say, “The image rotator was optimized on this day,” or “We changed to WebP images on this day when available,” and that way they can kind of tell when something got better and see if the business analytics did better after that time.

Tim:

That’s one other thing that you brought up in the email, was that you’ve done a bunch of experiments like the WebP formats. Do you have any cool stories or cool impacts to discuss on some of those experiments? Like for example the WebP experiment, how did that go and what was the takeaway there, or the gain?

Matt:

Yeah, it was really interesting. A friend at work and I wrote a blog for David Walsh about WebP images and what we’ve found in our experiments. We wanted to look at image quality and we wanted to look at implementation of how to use WebP today. What we found was that there are certain times and certain images that it really does make a difference.

So, one of our clients recently that—I’m project manager at Aristotle and so I do a lot of project management but then kind of as a side deal I do the performance stuff as well—and so one of the clients that I’ve been working with had this main page rotator, and it had some parallax on it, which is not really the best for performance, but we’re working on that too. But we had this image, it was a ping image that had this fade at the bottom of it, and it was like 600 or 700KB, and so I had approached our designers about WebP and was showing them the data that Google had given. And even looking—eBay had been using it; we were using WebPageTest to kind of test eBay to see how it works for them. And the designers really jumped on this and really loved the idea of WebP. Not being a designer, I know very little, but they talked a lot about the alpha channel, so I heard that thrown around a lot.

So we tried WebP and that 600 to 700KB image for this particular instance went down to, like, 50KB, and they were just amazed. So, they were looking at image quality and there were different places where the artifacts were different. But overall it went really well for them, their tests went really well. And of course then after they were on board, I had to go to the developers and say, “Hey, WebP isn’t supported by anything but the Chrome browser and the Android for Chrome browser, but we’d like to start using this.” So, that brought about the change of using the picture element, which is something we weren’t using here, and so there have been a number of discussions about the picture element and how to implement that.

I did a study where I did four different use cases where I loaded just a normal JPG and then I loaded the WebP using the picture element with a JPG fallback, and then I also used server-side htaccess code to load the WebP when available and a couple of different tests on how to implement it. There was also a WebP polyfill that we used, and we actually found the WebP polyfill would actually serve the WebP image to all the browsers—to every browser. The only bad part was that on iOS, it downloaded the WebP image three times for some strange reason. So, we’ve been looking into that and we’ve kind of come to the conclusion that the picture element is kind of the way to go, and so we’ve started implementing the picture element in our new projects for our new clients and are coming up with a plan to implement it for our existing clients as well.

But overall, it’s been very good, and of course we had to show that it was a worthwhile task to start doing all of this and so we did a lot of testing; some testing on an existing client to show that it was making the web page perform better.

Katie:

Do you have any data offhand of what you’ve learned in those experiments?

Matt:

I don’t offhand. A lot of the information that we have—and of course from the design aspect, my friend Adrian would be able to talk more about that; I did a lot of the implementing tests. But that David Walsh blog talks about everything that we found in that, and so I don’t know if we can put that in the footnotes or something.

Tim:

Absolutely.

Matt:

But yeah, as far as implementing it, like I said, we found that the WebP polyfill actually did fairly well and it implemented WebP for all the browsers. But in terms of future-proofing, that’s not going to be the best route. So, what we’re planning on doing is using the WebP polyfill for now, but we’re moving towards the picture element and using it through that. And really, once Firefox or Safari or Microsoft Edge, if they jump onto WebP, then we definitely want to start implementing the picture element and not use the polyfill at all. So, that’s kind of our plan of attack for right now.

Tim:

What a huge example of how why it’s so important to have everybody involved on the project invested in performance. I mean, it sounds like a simple experiment, like an innocent question in the beginning, “What happened with your WebP experiment?” Hearing how you had to make a case for it by tying it to analytics, you had to get the designers get on board with that, you had to get involved with the developers, and then there was the follow with the picture element. I mean that, on the surface, seems like a simple change but everybody involved in the projects are touching these things, which is why you have to get everybody together.

Matt:

Yeah, absolutely, and it’s important especially because as a developer once I start doing something, I kind of get in that mode of “Okay, this is how I’m doing it” and so it’s just been a process of—especially with the developers—having them say, “Hey, let’s experiment with this,” and letting them really run with it, not leaning over their shoulder and saying, “Here, now do this, do this.” It’s really letting them run with it and experiment a little bit and have that time, and then they come to that conclusion on their own of, “Yeah, this is actually really cool,” because now we can serve up responsive images, and it kind of caught on like wildfire once they started working with it.

Katie:

So I know that you said you can’t speak as much to the design-side of it, but what were some of the other exciting things that clicked for designers about WebP or just any of this performance stuff in general?

Matt:

Yeah, one of the things I do remember that my friend Adrian was talking about was that whenever you compress, take the quality of a JPG down really low, it was kind of blotchy in terms of there were some hard edges, artifacting. But on the WebP image when you took the quality down pretty low, it was actually kind of smooth and it smoothed out those hard edges and those lines, and he was excited about that.

One of the things we found that WebP didn’t do so well on was keeping textures. So, very detailed textures tend to not work for WebP. But in terms of having an image of a person with a sky background and they’re doing something, they’re at the mall or something and compressing that image, it did really well. And, of course adding an alpha channel, it worked really well in terms of compressing with the alpha channel as well. And then the lossy compression held up on its own, it did fairly well. And then lossless reduced the size of the image quite a bit.

But the biggest performance wins that we found with WebP were kind of the hero images that you would use on the main page of a website. Now in terms of smaller images that are already 100KB, there wasn’t really much improvement we found. But on those big hero images, you might be saving hundreds of kilobytes using WebP, so.

And it’s really, in our process and the way that we want to approach it, is kind of on a case by case basis. It’s not just like a one size fits all, but a designer talked about it as another tool in the arsenal. You just have to do some testing and really see, “Is this a case for WebP or not?” and then kind of go from there, and then the artist tells the developer, “Hey, these images are going to be WebP,” just so that they know how to implement them.

Tim:

Along the design-side of things, you had talked a little bit to us ahead of time about some of the stuff you were doing around perceived performance and actually measuring it through SEO-related analytics, which is interesting. Can you kind of dive into that a little bit? Because perceived performance is a big thing right now, but it’s also a young thing that we could use a little bit more information on.

Matt:

Yeah, absolutely. One of the things that we are experimenting with right now is when we make these performance optimizations, what effect does it have on SEO and how long that people are staying on the page, and how many visitors, does it increase traffic, does it increase time on page. And so we’ve kind of been taking that approach of using WebPageTest—we have our own private instance here—and using it to look at the video and the frame-by-frame shot, just to kind of see what the user experience is like as the page is loading. And so from there, we found there was one client that we had that the main page image rotator at the top of the page—the images weren’t loading until after everything had loaded, and for whatever reason this had happened. So we said, “Well, let’s take a look and change this so that the images load at the very beginning instead of after everything else loads.”

And so we did that and the Speed Index for the site dropped from like seven seconds to two seconds, and so there was this dramatic change. I told our marketing and SEO team, I said, “This is the date that we did this. Let’s take a look over the next few weeks and see if it helped at all.” And it actually did. It reduced the bounce rate for the website and it also helped the total time on the page and how long they stayed there. It was pretty interesting to see that and that it made quite a bit of difference for their analytics. So it’s something that we’re still working on, that we’re still trying to play around with a bit. But it’s definitely interesting.

Of course now with the perceived performance, we’re taking a look at loading above the fold CSS first and then the other CSS later and I’m seeing if that makes a difference. And so it’s been a lot of fun to be doing these experiments and seeing what happens.

Tim:

Wow, that’s awesome. So you took it from, like, 7,000 to 2,000 for the Speed Index. And what was the bounce rate gain? Did you have a number with that or no?

Matt:

I can’t remember the number off the top of my head, but I think it dropped I want to say 7% to 10%, somewhere in that range. So it did enough to make a difference and for our team to notice. We were very excited about that. And of course then we’re also wanting to make sure that nothing else is happening during the same time that we made these optimizations to make sure, and we didn’t see anything that happened along the same timeline as that. So, we’re kind of structuring these tests now so that we make sure that there’s no other factors that go into this, and it’s paying off quite a bit.

Tim:

Very cool. I would say, Matt, by the way that if you haven’t already considered doing so, that would be an awesome article case study to write. Just throwing that out there.

Matt:

Yeah, we’re actually working on that right now.

Katie:

When we talked earlier before recording, you had mentioned some of the pitfalls that you had encountered. Can you talk a little bit about some of the struggles and how you overcame that? Because I think they’ll probably resonate with a lot of people that are trying to do this now as well.

Matt:

Yeah, we had a number of pitfalls that came about, and of course some of them were performance-related, some of them were kind of team-related and getting everybody on board to caring about performance. But one of the things that we do at Aristotle is of course have a CMS for our clients to manage the content of their website. And so we tried finding a new CMS that would be able to handle tons of data because we have, especially on our tourism websites, tons of data coming through these websites with all the listings and different information.

So there was this one particular CMS that we tried and we kind of put all of our eggs in that basket and it ended up not working out; the data did not hold very well at all. So one of the things that we’ve been doing over the last several months is trying to find a CMS that can handle all of that data and still perform well. The problem with the last CMS was that it injected a lot of code into our code. We basically had to break it in order to get it to perform well, and even then it wasn’t performing like we wanted it to.

So we’ve been through the process of finding this new CMS and we’ve found a couple: one is Umbraco and one is Kentico. They’ve done very well in just getting out of the way and letting our developers do what they do best, and it doesn’t inject any code and so we have complete control over the code. In terms of performance, we’re able to really structure the site like it needs to be instead of having the CMS get in our way. So, that’s been one pitfall that we’ve kind of come up against.

I would say to anybody that’s kind of looking for a CMS and if you’re at a company that has a CMS for your clients, be sure that you ask those questions of does your CMS inject any code, is your CMS performant on its own, and how does it handle the data, making sure that the calls that the database are structured well. So, that’s one pitfall.

Of course, the other thing that I talked about earlier was just getting everybody’s attitudes on board with knowing that performance was a big issue. And it really helped a lot seeing these reports—Google got caught with the slow tag and all of a sudden everybody was like, “Oh my goodness, what is going to happen?” Of course, we don’t know what Google is measuring that off of, but we know that site performance is going to be a factor in search engine rankings, and so that really resonated with the company and it really resonated with the people that were higher up in the company, to say, “Hey look, in the future there could be this big red tag that says ’slow’ and discourage people from using our sites. We’ve really got to start now and get on the ball with making sure that all of our sites are performant and where they need to be.” So, those are just a couple of the pitfalls that we’ve run into over the last several months.

Tim:

Yeah, that’ll be a huge thing if Google does indeed end up labeling the search results like they’ve threatened.

Katie:

Well cool, Matt. Thank you so much for joining us. This has been really insightful for learning how your agency has sort of adopted all of these performance practices and your experiments look really cool, and we’re looking forward to your blog post about it. Is there anything else that you want to share about your performance wisdom and experiences?

Matt:

Um, yeah, performance wisdom… I’m just learning as I go and I enjoy the community—there’s a web performance community—and of course the community you guys started with that Slack has just been phenomenal. I love just sitting in there and listening and talking with others. Just the other day I was talking to Andy Davies about some WebPageTest private instance stuff.

So, I would say just keep experimenting. If you’re getting into performance and you don’t know where to start, just start trying to make your site faster, start looking at the different blogs and jump into the performance slack and check out WebPageTest.org. I mean, that’s just kind of how I got started. And then from there, I just kind of started asking “What would happen if I did this?” and then I just started writing different blogs about it and sharing what I learned, and that’s what I enjoy doing.

So, just keep experimenting, keep sharing what you’ve learned. One of the things when I started doing blogs was I was like, “Well, nobody’s going to want to read this.” But something that somebody once said was, “You may not think that anybody is going to want to read it, but there’s tons of people behind you that don’t know this stuff that are going to want to know this information,” and so just start blogging, start writing about what you’re doing and start sharing in the different communities.

Tim:

Great. So Matt, if people want to follow along with you and the performance improvements you are making, where can they do that?

Matt:

Yeah, I’m very involved on Twitter, so I tweet a lot—I think I retweet more than I tweet, just from the different resources. One of the things I do in my spare time is I teach front-end development and AngularJS through a company called Thinkful, and so my Twitter is kind of I tell people to go there to find different people to follow on Twitter so that they can start getting involved in the community. So the people that I follow there are a lot of developers and a lot of people that are creating the standards of what we’re doing today. So the twitter handle is @TheMattShull. Also, I do a lot of blogging. I don’t have a blog but there’s different places that I write to. I’ve done a couple of posts on David Walsh’s blog, one about WebP images and the other about measuring web performance, so those are up as well. Then I’m going to be speaking about web components and even a little bit about performance of web components June 2nd at Future Insights Live in Las Vegas as well, coming up.

Tim:

Very cool. And Katie will be there too actually, so it’ll be a performance powwow.

Katie:

Yes!

Matt:

Yeah, I hate that we’re talking at the same time though because I really wanted to see your talk.

Tim:

Multitrack events, ugh…

Matt:

[laughs]

Tim:

Anyway. Well thank you, Matt. That’s fantastic.

Matt:

Thank you guys for having me. I really enjoyed it.

Katie:

Yeah, thanks for joining us.

Tim:

Thank you for listening to this episode of the Path to Performance podcast. You can subscribe to the podcast through iTunes or on our site, pathtoperf.com. You can also follow along on Twitter @pathtoperf. We’d love to hear what you thought, so feel free to drop us a note on Twitter or leave a raving and overly-kind review on iTunes. We like to read those. And if you’d like to talk about being a guest or sponsoring a future episode, feel free to email us at hello@pathtoperf.com.