Why I Stopped Using Facebook’s Mobile App

Over the past 4 years, I have gone through periods of installing and uninstalling Facebook’s mobile app out of frustration and protest because of reasons you are all likely familiar with: 
  • why is my battery dead!?
  • why is my timeline so messed up!?
  • why is my private post viewable by everyone on the internet!?
  • why do these these security settings keep changing!?
  • why is the mobile app nearly 1/2 a GB!?
Facebookers will find these complaints reasonable, but, probably not compelling enough to stop using Facebook’s mobile app. They have their reasons, I have mine.
My most compelling reason to use Facebook is communication with my immediate and extended family because it is their preferred method vs. the telephone.  
But this past winter I found my final reason to stop using Facebook’s mobile app: it was listening to my private conversations!
 
Despite numerous claims to the contrary, this post is about my experience with Facebook targeting ads I believe were based on conversations held in proximity to my mobile device.

The Incident

Last year my wife quit her job and went back to school to study a new field, one that I was not familiar with or personal interest in.
 
In January, she and I sat in our living room talking about her day at school with my phone on the table, in the open, between us. She excitedly told me about a new company she just heard about that she wanted to purchase products from to help her start her business. The company name she mentioned immediately struck me as unique, so much so that I repeated it in a silly voice because it was phonetically fun to say.
 
About a day later, I logged into FB using my desktop browser and the very first advertisement presented to me was an exact match for the company my wife mentioned in our conversation. I recognized it instantly because the advertising was for a category of products that I have absolutely no personal use for, and, the name stuck out to me as being unique.

The Test Environment

This information is also pertinent to my findings:
 
The hardware (in proximity to conversation):
  • My iPhone
  • My wife’s iPhone (no social media apps or accounts)
  • Samsung TV (audio features disabled)

The system software and applications:

  • Facebook mobile app – installed on user’s device, placed on a table between user and another during conversation.
  • Facebook platform – highly available processing and storage tier
  • Facebook web app – accessed via laptop, in a different room on a different floor the house.

The user (himself):

  • software engineer with 20+ years experience verifying systems and applications using automation
  • one-time analytics dashboard developer for presenting insights into content targeting effectiveness.
  • one-time commercial broadcast radio voice-over artist for radio advertisements and promotions.
  • long-time meta-forward-observer and mostly hater of ALL advertising (except the 0.001% that are actually well written or funny). 
  • curious, data-driven do-gooder, snark
The audible data:
  • company name was not a ‘dictionary’ term found in regular or common speech.
  • company name was a mashup of two words (likely for distinction and because the web domain was available for their digital marketing).
  • company name was easily distinguishable from other common phrases my wife had been using during her course of study.
  • company name was never once previously mentioned in our private conversations
The advertiser:
  • does not have radio or television advertisements broadcast in my area (or on any streaming services we use)
  • markets to clinicians
The platform: 
  • I have a Facebook account
  • wife has never had a FB account
  • wife is not in my interest graph 
  • company sells products I have zero interest in and never sought

The Methodology

How I noticed I was targeted was simple:
  • my wife spoke aloud a unique word (signal input)
  • in a conversation with my phone present (sensor)
  • which had Facebook software installed (platform client)
  • the word was never previously spoken or searched for (high cardinality, not in my graph)
  • platform matched advertiser keyword with my input (signal processing)
  • platform served me an ad via desktop web browser (web application client).
My “accidental” black box test of Facebook’s advertising system indicates the mobile app software is listening to my conversations. This methodology is called Observability (a measure for how well internal states of a system can be inferred by knowledge of its external outputs) which is a topic in Control Theory, which is the interdisciplinary study of dynamic system behaviors. Less formally, this means that from the system’s outputs it is possible to determine the behavior of the entire system.
 
To investors, I believe Facebook’s interest graph content targeting system appears to be working as designed, and, quite well. You should be pleased.

The Background

My job for the past 20+ years has been to observe and verify applications and their systems work as designed.
 
In my particular engineering role the single most critical skill to develop and hone is observing every action (and reaction) of a system while operating a software application. After two decades, it becomes a near constant state of mind which is how I connected these dots.
Caveat: I could be wrong. I have not done the forensic analysis of the code or the devices. I am writing this based on my many years of experience finding and isolating system behaviors like this one I observed. I’m also relying on numerous sources of published statements and research material to support parts of my claim.  

The Material

Last year Facebook said it does not use your phone’s microphones for ads or newsfeed stories but that was after numerous reports came out that they had been listening.
 

Privacy Policy on Red Button Enter on Black Computer Keyboard.

Facebook’s Privacy Policy Device Information section describes how they fingerprint my device, including it’s location (especially in relation to “specific” locations) using GPS, Bluetooth and WiFi, depending upon permissions I’ve granted.

“We collect information from or about the computers, phones, or other devices where you install or access our Services, depending on the permissions you’ve granted. We may associate the information we collect from your different devices, which helps us provide consistent Services across your devices.” 

Even though the professor’s claims (links above) have been debunked, Facebook admits it did but now “only listens for 15 seconds while you are posting.”

So why continue writing this article?

  • I was not posting during my conversation with my wife (phone was not in my hand)
  • I intentionally removed FB app access to my device’s microphone (long before this incident)
  • I am also pretty darn sure I was logged out of the app. 

Additionally, it’s possible to “listen” to audio within proximity using other device sensors besides the microphone.

The Technology

Before you go thinking I have a collection a tinfoil hats for every occasion, there is a plethora of material produced by much more qualified researchers and engineers than I on the subject of tracking users via mobile phone sensors for content targeting. They have published, presented and even demonstrated on the subject. They are also questioning why device makers allow apps direct access to sensors. 
 
One of the most interesting and compelling technical sources I found when I started researching direct access to hardware sensors via applications came from a Stanford University paper which describes how mobile applications are given direct access to hardware sensors. In particular, their research proved your phone’s accelerometer can ‘listen’ to your speech. 
 
– Project with Videos (very nerdy!): https://crypto.stanford.edu/gyrophone/
 
What is important here is that it’s not your microphone (the obvious choice that requires your approval) being employed to “listen”, it is that ‘other’ non-obvious sensor with the funny name that is carrying the water here.
 
To give you a metaphor, your device OS governs whether or not applications can acquire access to sensors, however, if  the application convinces the user for permission to use a sensor, the user can override the OS setting. That’s like the irony of the FDA requiring labels showing active or regulated ingredients – there is no law to requiring non-regulated (aka unknown and unlegislated) ingredients to be listed. More importantly, the FDA is not always up to date on the latest technology, manufacturing processes or impact on the consumer a food or drug may have.   
 
Where things get a little scarier is how some applications are able to obtain direct access to a sensor without any further governance.
 
To clarify my point, this isn’t Jason Bourne/Snowden stuff. This is all squarely in the realm of technology used today by web services and social media for content targeting. 

The Market

As I mentioned above, digital advertising is the primary revenue engine on the internet – you (unwittingly) give up your privacy to post cat pics and they return the favor by trying to sell you Jucero and artisanal mouthwash…and you get none of the proceeds for having also providing your attention span. Of course there is a subscription revenue engine too but content “firewalls” have proven to be daunting to implement in a way that is both usable and converts users into subscribers.

To start simple, eMarketer estimates that Facebook’s platform will generate $36 billion in net digital ad revenue in 2017, up 35% from last year, giving it the second largest share of the global online ad market with 16.2%, behind Google’s 33% (double FB’s take). 

High volume ad targeting platforms like Facebook and Google are not unlike the scheme in Superman 3 when Richard Pryor’s lovable character hack’s his employer’s mainframe to collect the salami-slices of a rounding down routine on everyone’s paycheck. 

 

Regarding the advertising “spend” (budget), one forecast for 2017 indicates the digital (internet) ad spend will surpass the television ad spend for the first time in history (click to expand table).
 
Yes, we’ll still have glitzy/campy Super Bowl TV ads to watch next February, but, the largest slice of the ad spend pie will go to digital – this is a big deal.
 
Why? Let me unpack this a little…
 
The overall ad spend by medium is typically broken down as follows:
  • Television
  • Digital (internet)
  • Print
  • Radio
  • Other
Now let’s look the American advertising industry medium timeline:

Here is the very first digital ad. Looking at the timeline, from an ad spend perspective, digital is experiencing warp speed growth over the other mediums, but that’s not the half of it. Tech has also convinced advertisers to trust them with delivering their ads to the right, best and most important eyeballs. 

The Cost

There is additional benefit to keeping the content cheap but that might affect each business differently.

The Scale

The scale of ad spending is massive.

This year’s total digital ad spend is expected to exceed $70 billion in 2017. A sub category of Digital is RTB (Real-time Bidding) or “programmatic” and it has skyrocketed over the past 2 years alone. This is the Superman 3 business model again, especially as it relates to automated high-frequency trading models. 
 
According to eMarketing, “nearly four of every five US digital display dollars will transact programmatically in 2017, totaling $32.56 billion. By the end of the forecast period, that share will rise to 84.0%, leaving little doubt that buyers and sellers are continuing to invest in automated ad buying.”  All this is despite a major controversy with Alphabet’s YouTube policies that saw major declines for content publishers.
 
The point is, social media companies invest millions in research and development of new technology so they can more accurately target you because it directly affects their bottom line – ultimately their raison de vivre.  

The Beneficiaries

And who is dominating in their ability to not only receive those dollars but make sure advertising feel they are well spent? Google and Facebook.  They are spending billions in research and development of not only the technologies they currently employ but also new technologies that don’t even have a market developed for them yet (like VR) so they can be there when our eyeballs and attention spans show up.

The Reaction

…has been mostly a deafening silence. 
 
Everyone has those moments when they ponder the tech swirling around them and what type of impact it is having on their lives or their children’s lives, but…
 
 
…just like that, your mind changed the subject and you moved on.
 
In 2017 a lawsuit that tried (and failed) to do something about this practice of tracking users after they log out (unrelated to snooping on conversations). 

“The decision, filed late Friday in California, gave Facebook a win in a lawsuit that accused the company of improperly tracking users’ Internet usage between April 22, 2010, and September 26, 2011, even after they had logged out of their Facebook accounts.”

Additionally, the decision said the plaintiffs failed to establish a “realistic economic harm or loss” stemming from Facebook’s comment – this is troubling.
 
The problem is this: the legal system typically quantifies damage, loss or harm in economic terms vs other more meaningful terms like privacy, integrity and transparency. 
 
Sadly, legal action failed to effect change, but, a company like Facebook must be just to keep their distant 2nd place spot on the ad revenue trough and their investors happy. 

The Conclusion

I quit Facebook Mobile apps the day I received the advertisement for the company my wife was so excited to tell me about. It’s been months and life without FB is definitely better than with it.

From this point forward, I will only login very ~infrequently~ from a desktop web browser (and not on a mobile phone) and only if and when I feel like it. My only real business to login is to check on some folks (I know they’re always ok already) or to help manage other pages and projects.
 

The Recommendation

My individual choices and how I explain them should not really matter to you – they are as much my opinion as they are true to me. What really matters is your own experience. 
 
I recommend that you consider doing your own deep dive into technology you use – do something crazy like…read the license and privacy agreements or the terms of service. Also, why not spend time (hours if necessary) learning how to operate your account on your favorite social network – it’s your account after all but do you know how to really use it? 
 
To me, knowing how the features of your favorite social networks work and affect what you see is pretty important. I also think it’s imperative to stay informed and make note of what’s really going on around you in your life. 
 
For example, as your own experiment, pay extremely close attention to what you like or comment on or share on FB. Understand what the subject matter of the content you interact with, understand who posted it, learn more about them, and, keep a simply journal of your actions: what day and time did you interact with someone or some content? what happened afterward? How is that different from before the interaction?
 
Watching how the machine responds based on your actions is one way to take some power back in this lopsided relationship.
 
Don’t be afraid, explore!

October Surprise Addendum!

  1. For those who think my 401k is based entirely on tin foil futures (it’s not), I have discovered there are potentially other factors that might have influenced how I got targeted with an ad for a company and product and interest completely outside of my graph, but, those other factors indicate methods that are flawed and to be honest, do not offer an alternative explanation. That said, here’s an article from WAPO describing the litany of advertising targeting options FB provides advertisers: https://www.washingtonpost.com/news/the-intersect/wp/2016/08/19/98-personal-data-points-that-facebook-uses-to-target-ads-to-you/?utm_term=.1081951348b0
  2. FB lives and dies by it’s algorithms and that became crystal clear in the headlines arriving after my blog post – unfortunately the story is more serious and potentially tragic for folks who believe in democracy. The ad targeting story is only getting started and will uncover not just the “evil” things done to poor unassuming voters in WI and MI (and 21 other states at last count), it will also uncover….features! Yes, features within FB’s advertising system. This “narrative” is undoubtedly important to Americans and supporters of free elections and the rule of law around the globe, but, what you are going to learn will be very small facts: some really smart folks outside of USA used features (aka algorithms) within FB’s advertising system to target US voters in the same way as Zappos and Zazzle target you with ads for shoes and t-shirts – the only difference is the motive and what is at stake (sales vs votes). But hey, if it takes some international intrigue to get my point across, I won’t complain (pass the popcorn).
  3. As a subtext, here’s a PBS radio broadcast about how FB ads allow users to target anti-semitic groups (who by the way are terrible): http://www.pbs.org/newshour/bb/facebook-allow-advertisers-target-anti-semitic-groups/. The investigative reporter in this broadcast describes how easy it was and clearly how little oversight there is in this system, revealing examples of techniques employed by forces outside of the US to influence the 2016 campaign. Hey, if an investigative journalist can design a campaign to reach hate groups, it’s not a stretch to see how an adversarial nation-state can automate advertising campaigns that mix-in stolen PII to influence the narratives users saw leading up to the election.  
  4. I recently did some more digging around into my FB settings recently and discovered a ton of crap in the interests and ads sections – interests and corporations/companies I had never interacted with or liked – there were some real head scratchers too. Sadly, what I found does not correlate to this story much (I didn’t find the company who targeted me in the list, but, I also didn’t find any thing even similar, leaving me to believe my findings above). TL;DR: it was just more WTF stuff that FB heaved onto my interest graph without actually getting a signal from me directly. On that note, I strongly suggest you check out your settings frequently to ensure there is no monkey business going on – might want to add a reminder in your calendar to do so once every 3 months.
  5. I still believe my phone was used to pick up audio to target me with that ad and despite multiple statements and lawsuits in which FB claims this is not happening, there is plenty of evidence to more than suggest (but in fact determine) that if they aren’t doing so any longer, they absolutely did at one time use microphones in your vicinity to target you with ads.
  6. Belgian Data Privacy Watch Dogs declare in court that Facebook is snooping on users, including people who don’t have Facebook accounts: https://www.bloomberg.com/news/articles/2017-10-12/facebook-is-watching-you-belgian-privacy-agency-warns-in-court
%d bloggers like this: