GDPR Compliance and Analytics

This post is more of a conceptual brainstorm about GDPR and still being able to acquire useful analytics data.

If I were to sum up the GDPR, I basically take it to mean that you cannot store/track data about your user without their consent, AND they have the right to be forgotten… i.e. they should be in control of their own data.  This basically means don’t track usage events using a user identifier (on iOS the venerable identifierForVendor property), as this would constitute gathering data about a person that can be inferred via this identifier.

If you can convince your user to opt-in, then great.  Business as usual just with some provisions to be able to delete any data they no longer want you to have/use.  But is there a way to still gather information about App usage without someone’s consent while still obfuscating who actually did the things you’re trying to track?  My understanding is that you can gather all the data you want so long as it’s not possible to trace that back to the user itself.

My thoughts are that if you as an analytics person are able to relax your requirements somewhat, I think it could be possible.  If you as an analytics person don’t need real-time updates on how your users have been using your app, but still want all the same insights, I think it’s still achievable…. if you have patience.

I think the solution is to keep all usage tracking on the device itself, then say once a month you upload it all in one go to some custom API endpoint that would parse all that data into usage stats, all without some user identifier.  The semantics of that are “There is a user – we don’t know who – who used the app in the following ways last month.”  Then you can see funnels.  Then you can see retention.  Then you can see all of those things over a time frame that is useful.

Currently, if we uploaded each and every event as they happened, you’d have no way to connect all those events to a user.  If you upload all of those in one go, with timestamps, you can still process all this data and get a picture of what a user does in a specific amount of time without caring about who it was, specifically.


Level Up: Test-Driven Development

Until very recently, I’m used to being that hired gun who parachutes in, kills all the work tickets, asks the important questions about the product, makes an app work as desired on a tight deadline to a reasonable level of quality given the constraints, then is out again, all for a reasonably good sum of money. It’s exhausting because these companies often have no established procedures.  So in addition to being Lone Wolf: Kicker of Asses, I’m training junior developers so they can assist me without slowing me down too much, I’m creating an Adhoc QA Department (what is severity, what is reproducibility, how to write bug reports, what is a end-user test suite so you know how to regression test, and why you should use a ticketing system instead of just popping by my desk to effectively pull the plug on whatever 50 things I had in my head), I’m having to interpret incomplete designs, pull the assets out of the Design tools (Zeplin, Sketch, sometimes Photoshop) because many designers don’t know what exactly you need anyway, poke holes and/or fill in the gaps with the UX, and of course manage upwards. Oh yeah, and also do “Agile Waterfall” development, which is borne out of companies who only really do Waterfall but want to be “hip” to the new trends and demand we do Agile (with no scrum master or really anyone who knows how to lead that effectively). So then your time is further taken up with meetings and pushing work tickets around that actually don’t really encapsulate the work you actually need to do, but managers need you to do that so they can generate reports with some data that is meant to impress other people who really have no idea what’s going on anyway, and actually just increased levels of trust in your hires and “we’re good” would be equally effective/ineffective. (Ah, perhaps they don’t know how to make the right hires or can’t get them if they did.) In all of that, I have to “get ‘er done” because the deadline doesn’t change and surprise! All of your dependencies (Design, Backend API) also have the same deadline.

Yikes. A day in the life.  That above would be the worst day in the life.  It’s rarely all of those things.

So I’m grateful for my current freelance contract. It’s the first contract really in years where I felt I’m working in a proper software development company. The management overhead seems low, but the organization is not impacted. They have a process here that works. (I think it’s just because I hit the jackpot and they just placed a priority on the members of the team; personable yet very competent. Ultimately it’s a team who cares about what they do, and making sure there’s a nice team vibe. It also helps that they have corporate backing and therefore have a generous timeframe and budget it would seem.)

“Take the time you need to do a good job.” This is very much the culture here. For one of the first times in my career, I’ve been really exposed  to an office environment where you’re given time to think, and time to write a lot of tests while you develop. You can ask for specifications and those exist and are fixed. There are 2 other iOS devs here to bounce ideas off of, and of course to do code reviews with. It is so extremely satisfying when you get to refactor your original approach into ever more concise code that is more robust and less error prone. Time where you can write the API docs and the Unit Tests to basically freeze the specification. Normally there just isn’t enough time, given all the other tasks that Lone Wolf has, AND the product design always seems to be a moving target.

In short, it feels like I finally have the time and space to level up. Unit Tests are especially awesome when you work in teams and I’m glad for the opportunity to work in this environment for a while so I can establish some good habits and really reach a new plateau in my journey as a software developer.

The Perils of Hackers in your Team

I’ve currently joined a project as the only iOS Developer.  I’ve inherited a codebase that’s passed through the hands of 4-5 guys over the course of 3 years.  3 years is already at least a yellow flag in terms of iOS Development.  In my view, the only reason the same project should be allowed to live for 3 years is if you have older devices to support, or a massive userbase and a fat client architecture (making loss of data an issue).  For startups, you should always consider re-writing your Minimum Viable Product, as so many lessons are learned, so many pivots are made, so many corners are cut, to get you to where your codebase is today.  Cruft (look it up) is your biggest problem.

Poorly structured code will hurt your bottom line.  It becomes increasingly expensive to work with, and adding new features will not be easy, and the chances of introducing bugs will be high.  As much as one can talk about Test Driven Development (TDD), I’ve rarely heard of teams (in the startup world at least) that write tests, and if they do, they write them *after* the code has been written.  (This has proven to be the most expensive way to implement tests).  So, fine, the world is not perfect.  In these cases I would hope that if you are a developer reading this, you will at least put comments in your code and also NSAssert statements (the poor man’s sanity check / testing shortcut).

Let me now define what I mean by Hacker in the context of this post.  A Hacker is someone who may look good to any non-developer.  He gets the job done.  The code “works”.  He appears passionate as he types away in a frenzy, trying to make things ‘just right’.   The thing about a Hacker is that he may be self-taught and completely lack the principles of good software design.  He writes spaghetti code (that works).  He may say things like “I don’t need to put comments, the code should speak for itself.”  (If that’s true it is saying “Help me!” or “I’m taking hostages!” or “Your time now belongs to me!”).  But the worst thing about a Hacker on your team is that he will write code that is often a testament to every anti-pattern you have ever known, and to have to work with that person’s code is like being sabotaged before you can even begin.  Because to work with that code, you essentially have to rip out everything he’s done to make the codebase useable again.  So before you can begin to be awesome,  you have to weather the storm of management (who can’t possibly know your struggles) saying “I’m noticing progress is slow”, or “but it’s working now, what do you mean you need to rewrite it…?”

In short, a Hacker in charge of your codebase is basically signing a death warrant for your current codebase, or paying for a huge financial hangover after he’s gone.  A Hacker on your team can have some benefit (thinking outside the box, enthusiasm), but be wise and make sure he’s not in charge of architecture, and make sure you do code reviews.