Apple Special Event and Security

@ WWDC

Apple’s facial recognition has spurred a number of discussions about the privacy trade-offs in the iPhone X.

Experts Weigh Pros, Cons of FaceID Authentication in iPhone X | Dark ReadingOne concern about FaceID is in its current implementation, only one face can be used per device, says Pepijn Bruienne, senior R&D engineer at Duo Security. TouchID lets users register up to five fingerprints. If a third party obtains a user’s fingerprint and reproduces it, and the user is aware, they could register a different unique fingerprint.

Can Cops Force You to Unlock Your Phone With Your Face? | The Atlantic – Even if Face ID is advanced enough to keep pranksters out, many wondered Tuesday if it would actually make it easier for police to get in. Could officers force someone they’ve arrested to look into their phone to unlock it?

How Secure Is The iPhone X’s FaceID? Here’s What We Know | Wired – Marc Rogers, a security researcher at Cloudflare who was one of the first to demonstrate spoofing a fake fingerprint to defeat TouchID. Rogers says he has no doubt that he—or at least someone—will crack FaceID. In an interview ahead of Apple’s FaceID announcement, Rogers suggested that 3-D printing a target victim’s head and showing it to their phone might be all it takes. “The moment someone can reproduce your face in a way that can be played back to the computer, you’ve got a problem,” Roger says. “I’d love to start by 3-D-printing my own head and seeing if I can use that to unlock it.” 

Now lets talk about the Apple Watch, which I consider to present more serious issues.
 
The Apple Watch 3 is interesting from a legislative point-of-view. The software SIM in the Apple Watch clones the number of your iPhone. The security services of the major powers generally don’t broadcast their capabilities. Politicians are generally untroubled by knowledge of what is possible. Giving politicians an inkling is likely to result in broad sweeping authoritarian power. 
Imagine what will happen when Amber Rudd goes into parliament looking for real-time access to everyone’s phones. She now can point to the Apple Watch 3 as evidence that LTE and 3G connections can be cloned. What kind of legislation will her special advisers start cooking up then?

Secondly, it will only be a matter of time before criminals either work out how to do it themselves, or co-opt mobile carrier staff. Two factor authentication that depends on SMS is already compromised. This allows it to be compromised and undetectable.

The Apple Watch 3 may have royally screwed us all.

Brandon Beck of Riot Games on eSports

Beck is the co-founder of Riot Games (best known for League of Legends) on the rise of eSports and what its future looks like.

Interesting that Riot are trying to give players a better base to build their careers, but how long is their professional life, when do they burn out?

Thinking about Marcel

Publicis Groupe announced two things in the past week that caught the attention of the industry:

  • Withdrawing for 12 months from all promotional activity spend including the Cannes Lions awards
  • A Groupe-wide 12-month digital transformation fronted by a personal assistant app

You can’t look at either in  isolation, they are both linked together.

Why the withdrawal from promotional activities?

There are various speculative takes on this:

  • Other groups doing better at Cannes Lions this year had caused them to ‘take their toys to go home and sulk’. I hadn’t looked at the Lion awards scores, but I wouldn’t think that this is the reason. Clients would react negatively to it. Clients have egos too
  • Cannes Lions have gotten too expensive. Running events on the Côte d’Azur has never been cheap. The hotels can charge premium rates, due to demand being greater than supply. The GSMA World Congress moved to Barcelona in 2006 for this reason. Cannes can still run a good event and the infrastructure is ideal for advertisers. Other groups like WPP have pared back their spend but not cut it completely
  • It’s designed to focus spend on the things that matter for the next 12 months. This was one reason articulated by Publicis. The spend involved isn’t going to make a significant difference. At least, not on a project of the scale outlined by Publicis
  • It’s designed to focus staff on the things that matter over the next 12 months. I think that this is a key factor. Marcel is a software layer for a wider culture change the ‘Power of One’. Forcing the agencies to work together to provide a full deep offering for the client. This creates an internal market for services, skills and knowledge. There is no use having a development team if you can tap into Sapient. This also leads to a de-duplication of capability, increase in efficiency (% billable time).  It also reduces duplication of knowledge creation – tap into it wherever it is. You would need to balance this against client confidentiality
  • It’s a PR stunt. If handled well Publicis could gain a lot of positive coverage from this. It’s a classic example of what Sun Tzu called ‘The Void’. It’s also a bloody expensive PR stunt – so one would have to presume this is a collateral benefit. What happens if Sapient doesn’t match what’s in the concept video 12 months from now? If it does succeed then Publicis ends up with a solution would help market their business – business eating its own dog food, as advertisement

Let’s move on to Marcel itself

It’s hard to deconstruct a corporate video to get a firm idea what the underlying form might be. The truth is that the underlying form may not even exist yet as a product brief. It takes time to coalesce an offering from high concepts to prototyping these concepts with a sampling of users. From then on you go to mapping out the functional requirements of the product and build it in a series of short sprints. Once you have a minimum viable product and tested it, you may want to tweak your project direction further.

However, when you dig into it, Marcel isn’t only about an app, but re-engineering most of the IT infrastructure as well in order to support the machine learning capability. Marcel will find it harder to learn if the data is fragmented in drives with different permissions, online services or even offline.

Carla Serrano describes Marcel as:

A professional assistant that uses AI machine learning technology across our 80,000 people in 130 countries to connect, co-create and share in new and different ways.

This won’t be like Alexa Home managing your calendar and your Spotify playlist.

AI is put in there for audience members who wouldn’t know what machine learning is. A nice succinct definition below via TechTarget:

Machine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. … The process of machine learning is similar to that of data mining.

Let’s tease out the functions

  • Connect – could be anything from an intranet directory to a social network a la Facebook Work. The key element for success would be to get people to complete their profile and for the content to be validated. From personal experience, it is best if you get people to do this right at the point that you are on-boarding them. Getting a mass-push on employees doing this would be a campaign of attrition since there is always a client call to do, pitch to write or creative concept to develop. The information could be pulled across from HR systems, business planning, time-tracking / accounting systems and scraping LinkedIn profiles but all the data will be sub-optimal. How do you ensure consistent quality data on staff expertise? The key benefit of machine learning would be pulling information capacity and personnel career ambitions alongside mining the profiles.  What I’ve talked about in this paragraph is a major undertaking of data integration in itself

I’ve ignored messaging as a function as most agencies use multiple channels for messaging including Slack, email, Skype/Lync or SMS. A messaging service might be built in, some of the interfaces could be ‘call-and-response’ chat bot style interactions.

  • Co-create – Co-creation could just be building a virtual team through the connection functionality, if its a platform in its own right what would that mean? Google co-creation platforms and you get 14,900,000 results. There are lots of options, opinions and descriptions of how to implement a platform to do it. Publicis could use some of these commercial off-the-self platforms. Decisions would have to be made if the co-creation would facilitate synchronous or asynchronous co-creation. Where do you want to have it involved in the process? Discovery, strategy, creative briefing, ideation, concept development? Is bolting Box.net accounts, Basecamp or Jira co-creation and where would the co-creation process benefit from machine learning?
  • Sharing – Back in the mid to lated 1990s knowledge management was a thing for technology marketers selling into enterprises. The idea was that a mix of data mining software (Autonomy or SAS Institute) would allow you to tap into the written knowledge across your company. Of course, it didn’t work out that well. Google tried a similar thing with its own Search Appliance hardware sold to enterprises. For a business like Publicis whose product is data, insights and ideas, the potential implications are huge

Based on Google’s Return on Information: Improving your ROI with Google Enterprise Search white paper here are some rough numbers that I came up with.

1706 - Marcel

The notional productivity gain is worth well over $400,000,000 in additional billable time, or like having almost 1,600 additional staff at little additional cost. The key word in all this is ‘notional’.

So what’s the downside to the factors outlined in the top-level view of Marcel?

  • Client confidentiality – imagine if you’re a client and you realise that your documentation within an agency can be searched for beyond the account team and could be used in ways that you don’t know about? This isn’t an unsurmountable problem, but it is something that I am sure Publicis would be thinking about
  • Changing working habits and culture – the most valuable files will be spread across Dropbox-like services, in email exchanges, on file servers, personal computers (Mac and Windows), USB sticks and optical media.  Software can look at unstructured data to try and make sense of it. But it needs access to the files first. As a manager how would you feel that you lose control over work assigned to your staff. How would you assess their work for their appraisals?
  • A marathon of sprints – this a huge IT undertaking across hardware infrastructure, networks and access. That’s before you’ve considered software development. On its own it would weighty task – in reality it will be a large amount of iterative tasks, any number of whom could delay or damage Marcel

Understanding the context for Marcel

The second half of the video is concept film of how Marcel would work in practice. It was likely put together to give voice to functionality rather than also thinking about tone. I would not be surprised if this was reused from an internal presentation to showcase the vision of Marcel to key stakeholders. The film has tonality in it is a bit concerning, I suspect it’s unintentional. If Marcel works as promised we would be in new territory for corporate culture however.

Having watched it reinforced to me:

  • The technical scale and ambition Marcel represents. It is a huge undertaking from a technical point-of-view
  • Marcel is just the start of the hard work for Publicis.

How do you ensure a culture that continues to attract and retain the top talent as the organisation gets Marcel operational?

  • What does it say to women (or men) who might want certain amount of work life balance due to family commitments or a desire to upskill?
  • How would it handle organisational politics?
  • Lesley might be requesting talent for his energy client but how would his demands be balanced against those of their line managers or other people in the business?
  • How might it redefine the role that line managers play for colleagues?

The partial removal of client services as a gate keeper between Jamie the client and Publicis talent was interesting. It would make client services job to get their arms around all the business opportunities in the client much harder. It would also be more attractive to certain clients who would feel more in control of their account.

Themes in the film:

  • Marcel is being used at night or in the twilight – usage massively extending the working day. Agencies aren’t really a 9 – 5 lifestyle at the best of times, but this video implies even less work-life balance as standard working practice. The introductory dialogue is shot at twilight and Alex the Asian American strategist, sits in an empty office at night time. Lesley is in the artificial time of an subway station and even the Arc de Triomphe dropped in is shot in twilight
  • Marcel is mobile – and being used out-of-the office in most of the film. This implies that the work day has no boundaries. Does it imply that mobile devices are no longer for reacting to urgent emails, has the balance of work expectations changed to zero-downtime always on proactive working? How would an agency team be able to keep their thinking fresh over the medium and longer term?
  • Marcel is desktop – Alex uses Marcel on a desktop computer and the web service provides a Statista like set of visualisations for data. The implication being a large amount of research source integration (social insights, market data, Kantar media data???). This would also affect third party licenses as information is pooled
  • The dialogue implies a ‘Siri’-like experience on the mobile app, except that it understands what you’re saying. Marcel is far more articulate conversationalist than Siri, Google, Alexa or my banks interactive voice system. He’d probably score highly on Tinder due having a personality. I suspect most of this is a plot device for storytelling. Alex gives voice to his key strokes and Marcel is manifested as a search box rather like Bing using a desktop computer. Lesley the South African client service person is not talking to his phone as he moves up the escalator – he is literally giving voice to his thoughts. He sounds stressed.
  • Jamie the client from a bank is an interesting vignette. She has direct access to Marcel as a client facing tool and it is suggesting Publicis contacts to her, normally you would expect a client services person to be that interface.
  • Ines, the copy writer in Brazil has the most positive experience portrayed. Marcel understands her complex career aspirations and offers her opportunities to work on an Indian project. It looks as if she is doing this work at home, again reinforcing ambiguous message on work / life balance?
  • All of the people are alone, Marcel is not shown being used in a normal office environment. Marcel becomes your team?

TL;DR

Marcel is the business equivalent of playing high stakes poker. If it is pulled off successfully it would put Publicis in an excellent position versus it’s competitors. However there is a lot that can go wrong from a technological and organisation perspective.

I don’t know how much of this can be realistically achieved in the 12 months that Publicis seems to have given itself? It strikes me that this is likely to be a transformation that would require much more time in order to fully match the vision outlined.  From a cultural perspective the challenge of ‘break, build, bond’ hides the level of complexity and change going on.

The biggest risk is what happens if Publicis doesn’t meet the wider industry expectations of success with Marcel? How will that affect client perceptions of them, or their ability to hire talent? How would it affect Sapient’s standing as a technology company?

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

SaveSave

Technology is way ahead of interface designs

This post has taken a while to write. When I started we were on the cusp of Apple’s Worldwide Developer Conference. If you’re interested in technology, but aren’t an Apple fan it still matters as it sets the agenda. Apple’s moves affect wearables, smartphones, tablets and OTT (over the top) TV services.
 
The New York Times published an interesting article Apple Piles On the Apps, and Users Say, ‘Enough!’.
 
Ignore the title of the article itself, which is a function of clickbait rather than content. Instead, it provides an good critique of interface design across platforms. It highlights:
  • The difficulty in finding and installing other apps inside Messages. Many users aren’t aware of the functionality. This is different to the ‘interface as oldster barrier’ that SnapChat had. DoorDash – a Deliveroo analogue dropped a support after a few months due to a lack of users. Apple took a second run at this with iOS 11 trying to improve discoverability
  • Apple 3D touch isn’t used to drive contextual features by app developers
  • The Apple Watch’s mix of crown, button and small touch screen made ‘lean in’ interactive apps hard. The Apple Watch interface isn’t learned by ‘playing’ in the same way that you can with a Mac or an iPhone. Apple’s forthcoming watchOS update looks to have Siri ‘guess’ what you want. It wants to provide contextual information to users (and reduce interactions)
If you ignore 3D touch for a moment, these problems are cross platform in nature. (Some vendors like Huawei have attempted a similar 3D touch feature in their own apps. They did not try to get developer adoption.)
 
Thinking about Messenger app developers struggle to integrate disparate features into the interface. The exceptions are:
  • LINE
  • WeChat – the take up of mini-apps in WeChat have been disappointing performers. Is this indicating a possible ceiling for functionality?
Wearables as a category looks thin, with Apple being one of the largest players. Pebble got acquired by Fitbit. Jawbone seems to be a dead company walking. Their blog was last updated in October 2016, Twitter in February. It’s ironic: their original BlueTooth headset business would now be a great opportunity.
 
I’ve tried Casio’s BlueTooth enabled G-Shock, four Nike Fuelbands and a Polar wearable. I am on my second Apple Watch and I still don’t know what the real compelling use case is for these devices.
 
So how does this stuff come about? I think its down to the process of creation, which affects analysis and critical analysis of the product. Creation in this case is essentially throwing stuff up against the wall until it sticks and then the process becomes reductive. As a case in point, look how smartphones have evolved to the slab form factor. 
 
Throwing stuff against the wall
 
I’ve worked enough times on digital products to understand the functionality is king. It’s the single most important thing. I’ve worked on products that wonderful functions but:
  • Consumers didn’t know they had a need, its hard to get consumers to build new habits. Forming habits can be hard
  • They were a bitch to sign up with. Yahoo!’s sign-up process killed products. It’s a fact. We’d get consumers hyped up, we’d deliver them to the relevant page and they wouldn’t convert. I didn’t blame them, if I wasn’t an employee or digital marketer I’d have done the same
That’s how products are now built. The focus is on speed of execution of the idea. It isn’t about thinking through the complete experience. Agile methodologies with their short sprints puts emphasis on function. Away from data to feed into big picture optimisation. A function focus means that you end up with ‘lean in’ interaction designs as default.
 
There aren’t many organisations that get it right. I’d argue that the early Flickr team and Slack ‘got it’. Though there are common factors:
  • Both Flickr and Slack had common key team members
  • Both products fell out of failure. Flickr came out of tools for Game Neverending. Slack began as a tool in the development of Glitch
Where are the ergonomists and futurists?
 
There are people who can provide the rigorous critique.
Back in the day organisations with large R&D functions like NASA and BT employed writers to envisage the future. Staring into the future became a career. People like Syd Mead provided a visual map of the future. Mead and others did a lot of work thinking about the context of technology to users. At the present time lots of criticism levelled at VR glasses is it being anti-social. This comes as no surprise to anyone who has read William Gibson’s Neuromancer. Social interaction is more likely to come glasses wearer to glasses wearer. It will happen in a virtual third space. Neal Stephenson explored this third space in Snow Crash. The Black Sun was a virtual night club.
 

Bill Moggridge, designer of the GRiD Compass computer – the world’s first laptop thought a lot about ergonomics. The laptop had a 11 degree slope from pop-out leg to the keypad. This is something that your MacBook Pro or Surface doesn’t have. There is a lack of depth in technology design compared to what Moggridge had. He brought in psychologists and studied human computer interaction. He eventually co-founded IDEO.

Whilst the elements that Moggridge looked at were well known the thinking doesn’t seep into product categories. We are very good at asking can a product be made. We are poor at asking what does the product really mean. Apple’s viewpoint on the tablet segment is a case in point.

The vast majority of tablets are used for lean back media consumption from watching films and reading books to reviewing emails. It can work as a productivity device in specific circumstances with custom built apps – say field sales or replacing a pilot’s flight paperwork. The keyboard and power of modern Macs (and PCs) provide a better tool for content creators; whether its analysing a spreadsheet or writing this blog post. 

Yet, since its launch by Steve Jobs, Apple has viewed the iPad as a new PC. The iPad Pro has been designed to try and catch up in features with the Mac. It is ironic that Microsoft has moved a slim ‘MacBook clamshell design’ analogue into its latest Surface range.  It is very different to the pragmatic design ethos of China’s ‘shanzhai’ gadget markers who came up with both laughable and smart solutions. Everything from the dual SIM phone to the phone / electric razor hybrid. Successes bloomed and oddities slipped into the night.

I watched Apple WWDC so you didn’t have to

There was a mix of hardware and software updates. Apple put a lot of focus on  virtual reality, augmented reality and prepping their operating systems for handling larger amounts of data.  There was work done to further optimise video and photo usage on device.

The event offered bad news for online advertisers and a number of consumer electronics manufacturers. Online advertising using retargeting or autoplay video is going to be blocked in Safari. The new HomePod speaker took aim at ‘casual hi-fi’ like Sonos, Bowers & Wilkins and Bose.

Apple is working very hard to try and understand user intent, which is one of the first pieces it needs to put in place to develop the experience of a truly programmable world. What do I mean by a programmable world? A ‘web of no web’ where device intelligence behaves as if it understands user intent like a good valet. It is moving in a stepwise manner towards this.

What was more surprising is how Apple has gone big on VR and AR creation and consumption. Whilst video post-production houses probably have the most to complain about when it comes to Apple’s Pro equipment, they are not name checked. Apple has started to move to address their concerns. The external graphics support in macOS implies that a furture Mac Pro will have the software to match hardware.

More details by platform:

macOS

The name High Sierra implied an OS update that might seem incremental to consumers, but has major technology changes under the hood.

  • Data – Apple File System as default (many features similar to Sun Microsystems’ ZFS). Faster for file swaps and giving a faster computer experience
  • Video – better quality video algorithms with smaller file sizes and integration with
  • Graphics – upgraded Metal API – Apple had been using it on machine learning applications within the OS. Metal 2 has been used to accelerate system level graphics and provides access to app developers. There is OS support for external graphics accelerators. The external graphics developer kit is based on AMD Radeon card.
  • MacOS supports VR through Metal for VR. Steam, Unity and Unreal supporting VR on the Mac. Apple seems to believe that VR and AR content is the desktop publishing of the 21st century, they have gone hard on making the best creators platform that they can
Safari
Focus on being the fastest browser experience, even in comparison to Chrome
  • Autoplay blocking – which will impact advertising network video views
  • Intelligent tracking prevention – positioned to target advertising retargeting and cross-site tracking
Mail
Productivity refinements including a split screen view
Photos
  • Uses machine learning to improve searching and photo recognition and integration with photo-editing

tvOS

  • 50 media partners integrated into TV app
  • Amazon is coming to Apple TV. Interesting move of detente between Apple and Amazon

iOS

iOS 11 – focus on underlying technologies:
  • Machine learning APIs – to help adoption of CoreML on device for third party apps
  • ARKit – to aid AR in apps. Clever work done on scaling and ambient light. This about providing a market for the content which which would be created on the Mac
  • Chinese specific features: including support for QRcodes, SMS spam filtering. Chinese users have a particular set of contexts and these innovations could become popular in the west
  • Interface tweaks in control centre and the lock screen.
Messages
  • Improving discoverability of app stickers and apps – much needed
  • Automatic synchronisation of Messages across devices, delete once, delete across all devices
ApplePay
  • Person-to-person payments as an iMessage app. Obvious competitor would be WeChat in China and PayPal in the west
Siri
  • Improved expressive nature of the voice.
  • Follow-up questions, presumably to improve context
  • Provides translation services
  • Siri integration into a wide range of apps including WeChat and OmniFocus They’ve tried to use on-device learning to try and improve context and being helpful. Siri knowledge is synched across devices. Uses web history to improve Apple News and custom dictionary spellings
Apple Maps
  • Indoor navigation for airports
Photos
  • Better image compression to save space on device. New depth API that can be accessed by 3rd party apps
  • Video autorotates a la Snapchat / Snap glasses
App Store
  • Apps now reviewed in less than 24 hours
  • First app redesign in nine years. Tweaks to improve discoverability and merchandising of apps including in-app sales
 watchOS
  • The biggest feature in watchOS 4 is the Siri-powered face. The Siri-powered watch face provides contextual information on the ‘home screen’. It takes into account past habits, time, location etc. Apple’s language around this was interesting, they described it as an ‘Intelligent proactive assistant’.

More details by hardware

Mac hardware
  • iMac – improved displays, brighter and support for 1 billion colours. Moving to Kaby Lake Intel processors. Up to 64GB of RAM on the iMac and 2TB SSD. Discrete Radeon graphics cards on larger iMacs. – big focus on VR development.
  • MacBook – Kaby Lake processors. Pro machines get updated graphics as well. The MacBook Air gets a processor boost.
  • iMac Pro – single piece machine with workstation specification including 10Gbit Ethernet. Presumably as an interim measure until the Mac Pro arrives next year. How upgradeable would the iMac Pro be, which is a key consideration for workstations
iPad hardware
  • iPad Pro – 20% bigger screen, 120Hz screen refresh rate. Doubling default memory sizes up to 512GB
 
Apple HomePod
Apple is going after Sonos and brown goods companies like Bose, Bowers & Wilkins and Bang & Olufsen. The Siri functionality is a hygiene factor rather than a serious competitor to Amazon Echo. There was a big emphasis on the privacy functionality of Siri in HomePod
Further reading

WWDC 2015: you know the Apple news, but what does it mean?
48 hours with the Apple Watch
Eight trends for the future: web-of-no-web
Eight trends for the future: contextual technology