Case Study - DocuSign Inc

DocuSign offers the world’s #1 e-signature solution, part of its cloud-based System of Agreement Platform. The company enables its more than 450,000 customers and hundreds of millions of users to eliminate paper and automate the agreement process for their businesses, helping reduce costs and largely eliminate errors.

Challenge

DocuSign wanted to improve the performance of its in-house landing page application. A lack of performance in this area led to user experience impacts, causing lowered conversion rates, higher customer acquisition cost (CAC), and lowered return on ad spend (ROAS). To overcome this issue, the engineering team began looking for the best strategy and solution: the company is large, growing quickly, and was experiencing the ever-expanding complexity within engineering that many successful companies do.

“The goal was “to improve performance”, but the real issue our team had was that we did not know how to approach this in a structured way, nor did we know how to effectively communicate improvements in a way that non-engineers could relate to.”

Greg Robbins, Sr. Director of Engineering, eCommerce at DocuSign

DocuSign’s VP of Engineering, Loic Houssier, sought outside expertise from Aster Labs CEO Mike Julian to provide the team with guidance on how to strategically approach the issues and make improvements that would have a business impact. DocuSign selected Aster Labs because they clearly understood the business needs and impact of the problem.

“Working through a consultant like Mike was a more efficient, faster way to achieve the results we wanted to see. It’s good to have someone with a lot of experience in various fields to guide you through. Insiders usually focus on the toolset and how it should be (better) implemented, rather than trying to understand the business and come up with the requirements of what you want to really measure. But it’s not about putting a lot of telemetry in your system. It’s about making it useful for the business. Mike has that business focus. And unlike some consultants, he’s able to use any tool and make the most out of what you have in place.”

Loic Houssier, VP of Engineering at DocuSign

Solution

Mike’s first step was to help the team understand their application and infrastructure from a performance perspective. Through guided group sessions, he walked the engineering team through an exercise to understand the communication flows and the necessary instrumentation points for the application. After guiding the team through the implementation of those instrumentation points, Mike conducted a full frontend and backend performance analysis using the newly-found data and made prioritized recommendations on changes to the application.

“Mike helped the team gain a better insight into the way our application works from a performance perspective, giving them a more detailed understanding of each element of performance of the website as well as what they need to monitor in order to improve the customer experience. This provided the team with a kind of playbook that they can apply to any of the applications we use when we’re facing an issue, which helps them be more efficient.”

Loic Houssier, VP of Engineering at DocuSign

Of course, seeing a one-time performance improvement is great, but creating lasting change is even better. To that end, Mike met with DocuSign’s major stakeholders to deliver the findings and recommendations. He conducted a Q&A session to help each stakeholder understand how performance helps their own department goals and how to truly make performance a feature and core part of the team culture.

“Mike helped us focus on a few key metrics and showed us different ways of measuring and presenting those metrics. He showed us not only how to make our sites faster, but also how to show that progress to the business in ways that are meaningful to other stakeholders in the company. We appreciated how dedicated, serious, knowledgeable and professional Mike is and his ability to get along with everyone.”

Greg Robbins, Sr. Director of Engineering, eCommerce at DocuSign

Results

With the improvements in monitoring and telemetry made as a result of the project, the DocuSign eCommerce team now has greater insight into the product’s inner workings. For each change they make to the product, the team can now see what impact it has on performance, and also allows them to spot and reduce technical debt.

The project led to a key business result for DocuSign: helping to convert more visitors to users of the platform.

“We have a definitive business outcome with tremendous improvement in performance and a better conversion rate. On some flows, we succeeded to reduce the time for certain transactions by 80% and we have seen an average performance gain of 50% on all the stacks we work on. And we are still seeing the benefits of Mike’s work with us as we keep working on implementing what he did with our team.”

Loic Houssier, VP of Engineering at DocuSign

Another significant outcome was the culture change within the team. The project led the engineering team to adopt more of a focus on performance in their work, including in QA efforts for new releases, ensuring they don’t lose the performance gains over time and are more aware of performance in their daily work

“Each time we ship a new version of the product, we now know that performance won’t be degraded and won’t decrease. We feel safer from that perspective. We’re also seeing our engineering team being more curious and proactive. When there’s something happening, even if it’s just a blip or just one outlier, team members dive into it to better understand what’s going on.”

Loic Houssier, VP of Engineering at DocuSign

The culture change extends beyond the eCommerce team, with key stakeholders across the company, from product, marketing, and more, now having a shared understanding and value of performance improvement.

The engineering, product, and business teams now all see performance not only as something we need to look at some point, but definitely as a feature that brings value for the customer that we need to spend engineering time on. Before, performance was viewed as an issue or technical debt. Now, it is an asset.

“Having Mike help us change the culture in the two months he was with us, it’s pretty impressive. I’d recommend him because he helps achieve business outcomes that make not only the engineering team happy, but also the business partners.”