GDPR Compliance and Analytics

This post is more of a conceptual brainstorm about GDPR and still being able to acquire useful analytics data.

If I were to sum up the GDPR, I basically take it to mean that you cannot store/track data about your user without their consent, AND they have the right to be forgotten… i.e. they should be in control of their own data.  This basically means don’t track usage events using a user identifier (on iOS the venerable identifierForVendor property), as this would constitute gathering data about a person that can be inferred via this identifier.

If you can convince your user to opt-in, then great.  Business as usual just with some provisions to be able to delete any data they no longer want you to have/use.  But is there a way to still gather information about App usage without someone’s consent while still obfuscating who actually did the things you’re trying to track?  My understanding is that you can gather all the data you want so long as it’s not possible to trace that back to the user itself.

My thoughts are that if you as an analytics person are able to relax your requirements somewhat, I think it could be possible.  If you as an analytics person don’t need real-time updates on how your users have been using your app, but still want all the same insights, I think it’s still achievable…. if you have patience.

I think the solution is to keep all usage tracking on the device itself, then say once a month you upload it all in one go to some custom API endpoint that would parse all that data into usage stats, all without some user identifier.  The semantics of that are “There is a user – we don’t know who – who used the app in the following ways last month.”  Then you can see funnels.  Then you can see retention.  Then you can see all of those things over a time frame that is useful.

Currently, if we uploaded each and every event as they happened, you’d have no way to connect all those events to a user.  If you upload all of those in one go, with timestamps, you can still process all this data and get a picture of what a user does in a specific amount of time without caring about who it was, specifically.

Level Up: Test-Driven Development

Until very recently, I’m used to being that hired gun who parachutes in, kills all the work tickets, asks the important questions about the product, makes an app work as desired on a tight deadline to a reasonable level of quality given the constraints, then is out again, all for a reasonably good sum of money. It’s exhausting because these companies often have no established procedures.  So in addition to being Lone Wolf: Kicker of Asses, I’m training junior developers so they can assist me without slowing me down too much, I’m creating an Adhoc QA Department (what is severity, what is reproducibility, how to write bug reports, what is a end-user test suite so you know how to regression test, and why you should use a ticketing system instead of just popping by my desk to effectively pull the plug on whatever 50 things I had in my head), I’m having to interpret incomplete designs, pull the assets out of the Design tools (Zeplin, Sketch, sometimes Photoshop) because many designers don’t know what exactly you need anyway, poke holes and/or fill in the gaps with the UX, and of course manage upwards. Oh yeah, and also do “Agile Waterfall” development, which is borne out of companies who only really do Waterfall but want to be “hip” to the new trends and demand we do Agile (with no scrum master or really anyone who knows how to lead that effectively). So then your time is further taken up with meetings and pushing work tickets around that actually don’t really encapsulate the work you actually need to do, but managers need you to do that so they can generate reports with some data that is meant to impress other people who really have no idea what’s going on anyway, and actually just increased levels of trust in your hires and “we’re good” would be equally effective/ineffective. (Ah, perhaps they don’t know how to make the right hires or can’t get them if they did.) In all of that, I have to “get ‘er done” because the deadline doesn’t change and surprise! All of your dependencies (Design, Backend API) also have the same deadline.

Yikes. A day in the life.  That above would be the worst day in the life.  It’s rarely all of those things.

So I’m grateful for my current freelance contract. It’s the first contract really in years where I felt I’m working in a proper software development company. The management overhead seems low, but the organization is not impacted. They have a process here that works. (I think it’s just because I hit the jackpot and they just placed a priority on the members of the team; personable yet very competent. Ultimately it’s a team who cares about what they do, and making sure there’s a nice team vibe. It also helps that they have corporate backing and therefore have a generous timeframe and budget it would seem.)

“Take the time you need to do a good job.” This is very much the culture here. For one of the first times in my career, I’ve been really exposed  to an office environment where you’re given time to think, and time to write a lot of tests while you develop. You can ask for specifications and those exist and are fixed. There are 2 other iOS devs here to bounce ideas off of, and of course to do code reviews with. It is so extremely satisfying when you get to refactor your original approach into ever more concise code that is more robust and less error prone. Time where you can write the API docs and the Unit Tests to basically freeze the specification. Normally there just isn’t enough time, given all the other tasks that Lone Wolf has, AND the product design always seems to be a moving target.

In short, it feels like I finally have the time and space to level up. Unit Tests are especially awesome when you work in teams and I’m glad for the opportunity to work in this environment for a while so I can establish some good habits and really reach a new plateau in my journey as a software developer.

No, I didn’t die

Looking at this blog, it’s been over a year since I wrote anything. I’ll just list my excuses:

  • I was overworked on some agency projects and there simply wasn’t enough time to write posts
  • It’s generally a grey area to write about problems solved for clients who make me sign an NDA
  • I could have made the solution to the problem look generic, thereby obfuscating my client’s identity and thus my attachment to the project, but didn’t.
  • And now THE main reason….

I spent 8 months of 2018 taking a sabbatical. I like to call it a sabbatical because this is how you don’t lose respect from people when compared to saying “I’m unemployed and am enjoying watching my beard grow”. But really it was more like a “I’m fed up with insane projects that are under-staffed, with poor management and deadlines that don’t change, so ultimately the bottom-feeding developer who is doing the job of a small team on his own is starting to wonder if it’s time to pack it in and open a bakery, or do some other kind of artisanal work that involves repetitive, meditative movements and a certain degree of craftsmanship. <blows sawdust off his handmade hope chest>”

Sabbatical has a nicer ring to it and doesn’t require a backstory.  “Ah, he must be a well-to-do kind of man.”

This sabbatical was not about going to some coding retreat so to level up my skills as a developer. It was more like a sanity check. The reason I like coding and software engineering is that I find it to be a highly creative medium. For me, being creative is what gives life some meaning and makes it fun. The process of having an idea, developing it in your mind, putting it down on paper, refining and iterating on that, solving the unknowns, then finally doing the implementation / building. Then it’s done, you pat yourself on the back, revel in your accomplishments, then say “next!!” I love it. So the sanity check was a) to get a little distance from a profession that was starting to become not fun for me anymore, and b) do something completely different but also creative.

So I did that. In 2018, I decided I wanted to build a camper out of some kind of vehicle, then drive around North America until I experienced some sort of epiphany. I gave myself a year. I used 8 months. I built a camper. I had an adventure. I drove 24,000km in 4.5 months and saw so many beautiful places and met tons of very lovely people. I had some profound realizations about my life that were restorative and empowering for me going forward.

In short it was the most useful year I’ve had in over a decade. If you have the opportunity to take a long period of time off and leave your normal surroundings behind, I highly recommend it.  You’ll return knowing what changes you need to make in life.

True, I have fallen a bit behind with the latest and greatest in iOS / Swift dev, but since I’ve been back working (since November 2018), I’ve been fortunate to work with great developers and I’ve actually leveled up in this short time. I can say I’ve become way more of a test-driven developer, and I’ve also got a few new tricks up my sleeve that you generally get to learn when you work with other developers that work on your platform; not something I’m all that used to as I tend to be either the only iOS dev on a project, or they quickly say “oh but you know the most”.

At any rate, I’m back and I’m still happy to be a developer, and now that my approaches have changed somewhat, I have new areas to grow. So I’m not stagnating like I felt I was, and I’ve been fortunate to get a few freelance contracts that have not turned about to be a complete and total clusterf*ck, which has often been the case in Berlin, Germany. I oftentimes have the feeling that when it comes to Software Development practices, Berlin is the wild west; there are no rules, OR Berlin is teenage love; just fumble around in the dark for a while and hope something good happens.

Core Data Migrations – Woes…

So thankfully I won’t go into a misogynistic blog post a la frustrated Mark Zuckerberg in the movie “The Social Network”, but sometimes you just need to blog to let your technical frustrations out!

I’m doing some complicated Core Data migrations where the lightweight ones won’t work.  I’m migrating from one Core Data model to a completely different one, and I’m trying to map those properties over.

I’ve already discovered a wealth of information either through google, trial and error, and a bit of both, which I will share once I’ve got the entire migration code running.

I just have to say here however… I think it’s probably best to avoid these Xcode tools like a mapping model.  Because they’re buggy as hell.  For example, if you define a mapping model and provide source and destination models, but then change either of those source or destination models, the Mapping Model will no longer work, and even if you re-select the source and destination, it doesn’t matter:  You’ve just killed your Mapping Model.  Is there any info about this?  No.  Just “could not find a suitable mapping model”.

Ugh.

I’ve been trying to debug something for 3 HOURS because all you get is an EXC_BAD_ACCESS crash, and ZERO information as to what it’s all about, and NSZombies do nothing.

So I’m sitting here, taking shots into the dark and hope I discover something.

I question whether I should just give up on TOOLS, which are supposed to make your life easier, and just write the whole heavyweight migration in code.  If I had started on that 3 hours ago, I’d probably be done.  My data model is like 8 entity types.

Is it just me or is the quality of Apple’s tools getting worse?  It’s like they produce stuff that’s 80% good.  But it’s a tool chain, so 0.8 * 0.8 = 0.64.  So you see where this is going…

Nostalgia for old iOS

This sounds silly, but I kind of miss iOS development in the days before iOS 7. I kind of have this feeling the entire ecosystem has become less stable, less reliable, not as clear in its vision, and basically less easy than it used to be.

Don’t believe me?  Just look at all the different ways Apple has tried to solve Autorotation.  That API has just changed so many times.

The thing that actually inspired this post is actually not related to nostalgia because it’s an issue that has always been less than straightforward.  All I want to do is tell my navigation bar that the back button should have no text (just the arrow).

There are all sorts of posts on this, but none of them actually work.  Change the title of the back button in viewWillAppear: ??  Nope.  Override the navigationItem variable and set its back button to an item with no title?  Nope….

Just sometimes simple things in iOS are annoyingly not that way.

Developing for iCloud sucks

iCloud is the thorn in Apple’s side.  They have been trying to get this right forever.  Maybe it works now from an end-user perspective, but as a developer, what a rough, confusing implementation.

I have to say, when you work with the Dropbox API, it is intuitive, it is easy, it is straightforward. Perhaps you have to write a lot of boilerplate code, but then again, do you?  It manages files.  This name, that path, that revision number, that modifiedDate.  You sort out the rest.  How hard can that be?

I’m currently writing a sync engine that allows users to decide which cloud service they want to use to backup their data.  So of course it would make sense to have one common API that wraps the various services.

It went so smoothly with Dropbox.  The only difficulty I had was realizing that the serialization of date strings sometimes results in small differences in the Date that emerges, meaning I had to write a .isTheSameAs(other: Date) method, which checks to see if they are within X seconds of each other, then call those ‘the same’.

Then I moved on to iCloud.  What a clusterf•ck.  Talk about over-engineering and complete inflexibility.  I just want a list of files that are in the cloud.  No, Apple has to distinguish whether you have them locally or not, and gives them a different URL.  And all sorts of other weird stuff.

So I tried to find a framework that makes iCloud a little more user-friendly, called iCloudDocumentSync which I found to be pretty straightforward, all things considered.  And yet, it’s still a pain.  Because iCloud uses multiple URLs to represent the same document.  In the end, I have to sync the entire iCloud folder’s contents before I can start working with these with a common URL.  (Sure I could extract the file name I need).  Moreover, iCloud uses a thread to open each document, so if I want to batch download a lot of data, suddenly I have a thread explosion.  Now I have to deal with managing that.

All in all it’s just been a pain to work with it.  It seems opaque in its verbosity.  Meaning, the person who talks a lot doesn’t get listened to and understood.  There are so many callbacks and all sorts of weird things going on, it’s very annoying.  Just because they’re trying to save the developer time, they’ve implemented something weird and unwieldy if you don’t use it according to their use case.  I personally just use a UIDocument as a means to save data.  I don’t work with UIDocuments because my app’s data model uses the data, but not the file.  This just seems silly with iCloud.

Alamofire and nostalgic feelings for AFNetworking

So, in the Swift world, I presume a lot of people prefer to remain “Swifty”.  That is, why would you opt for old Objective-C libraries?  Out with the old, in with the new!

So, we adopt Alamofire.  Is it just me, or is this library a bit unwieldy?   I think it has a lot to do with Swift shorthand and often not knowing just what types are actually being passed around, especially given autocomplete.

On top of that, just when you get your head around a specific release, they make a new major release, breaking old APIs.  And if I’m going to have to keep updating my code to stay current because “out with the old, in with the new”, why don’t I just stay with AFNetworking?  I mean come on.  It’s networking.  For the mostpart it’s just “give me that data at that URL.  Here’s my auth info.”  Done.  Or “here’s some data for you.  Here’s my auth info.”  Done.

Anyway, it’s a rant.  I just don’t find Alamofire all that sexy.  It reminds me of dealing with civil servants.  The creators imagined this bureaucratic utopia that functions perfectly as long as everyone understands it.  Furthermore, we must not only understand it, but are fully on board with its vision.  Meanwhile, we the people are busy trying to write our own apps, and couldn’t really care less.  We just want to get data and post data and not have to deal with too much crap in the middle.

(Go ahead, snarky programmer.  Now tell me off, tell me to just use this, that, or the other.  Reject my feedback.  It’s fine.  All I’m saying is that AFNetworking seemed a lot easier to use.)