Facebook will assemble an independent Ethical, Legal and Social Implications (ELSI) panel to oversee its development of a direct brain-to-computer typing interface it previewed today at its F8 conference. Facebook’s R&D department Building 8’s head Regina Dugan tells TechCrunch “It’s early days . . . we’re in the process of forming it right now.”

Meanwhile, much of the work on the brain interface is being conducted by Facebook’s university research partners like UC Berkeley and Johns Hopkins. Facebook’s technical lead on the project Mark Chevillet says “they’re all held to the same standards as the NIH or other government bodies funding their work, so they already are working with institutional review boards at these universities that are ensuring that those standards are met.” Institutional review boards ensure test subjects aren’t being abused and research is being done as safely as possible.

Regina Dugan presents at F8

Facebook hopes to uses optical neural imaging technology to scan the brain 100 times per second to detect thoughts and turn them into text. Meanwhile, it’s working on “skin-hearing” that could translate sounds into haptic feedback that people can learn to understand like braille. Dugan insists “None of the work that we do that is related to this will be absent of these kinds of institutional review boards.”

So at least there will be independent ethicists working to minimize the potential for malicious use of Facebook’s brain-reading technology to steal or police people’s thoughts.

During our interview, Dugan showed her cognisance of people’s concerns, repeating the start of her keynote speech today saying “I’ve never seen a technology that you developed with great impact that didn’t have unintended consequences that needed to be guard railed or managed. In any new technology you see a lot of hype talk, some apocalyptic talk, and then there’s serious work which is really focused on bringing successful outcomes to bear in a responsible way.”

In the past, she says the safeguards have been able to keep up with the pace of invention. “In the early days of the Human Genome Project there was a lot of conversation about whether we’d build a super race or whether people would be discriminated against for their genetic conditions and so on” Dugan explains. “People took that very seriously and were responsible about it, so they formed what was called a ELSI panel . . . By the time that we got the technology available to us, that framework, that contractual, ethical framework had already been built, so that work will be done here too. That work will have to be done.”

Building 8 R&D division head Regina Dugan at Facebook’s Area 404 lab

In just the span of a week, Facebook went from being criticized for not innovating and just copying Snapchat, to merely using its social network monopoly to squash the innovation of others, to innovating so far into the future that it scares us and conjures dystopic thoughts.

Worryingly, Dugan eventually appeared frustrated in response to my inquiries about how her team thinks about safety precautions for brain interfaces, saying “The flip side of the question that you’re asking is ‘why invent it at all?’ and I just believe that the optimistic perspective is that on balance, technological advances have really meant good things for the world if they’re handled responsibly.”

Facebook’s domination of social networking and advertising give it billions in profit per quarter to pour into R&D. But its old “Move fast and break things” philosophy is a lot more frightening when its building brain scanners. Hopefully Facebook will prioritize the assembly of the ELSI ethics board Dugan promised and being as transparent as possible about the development of this exciting-yet-unnerving technology.

Facebook Spaces was announced at the social network’s F8 conference as a way of blending social media and virtual reality. If you own an Oculus Rift (and Touch controllers), you and four friends can enter a virtual world and hang out together.

Unfortunately, hanging out mostly constitutes of chatting, taking “selfies*” and enjoying the virtual world around you. Oh, and it’s not you, per se, but a cartoon caricature that you control; a dead-eyed digital mannequin that smiles blankly at everyone in the desperate hope that you won’t find it creepy.

Unfortunately, Spaces falls into the same trap that so many other platforms have over the years, assuming that people want the digital world to be a surrogate for the real. And that people have the time and energy to sit around talking to a cartoon with their friend’s voice through thousands of dollars worth of equipment.

Whenever a high-minded tech company makes proclamations about the future of social interaction, I’m reminded of Second Life. Founded more than a decade ago, the title was an open world platform that enabled strangers across the world to come together. It was touted as a digital utopia, a place where you were free to create whatever you wanted, whenever you wanted, however you wanted. Your digital avatar could even fly around the Second Life universe, interacting with people from across the globe.

But in reality, this utopianism was misplaced, and the environment, even during its boom years, was stale and empty. People weren’t all that into into communicating with each other, and the platform gradually became a space for people to develop weird forms of architecture. It became little more than a proto-Minecraft, a gallery space or weird and wonderful designs that few would ever notice. Second Life itself still exists, of course, with around 600,000 monthly active users.

A big part of the internet’s job is to bring people together in ways that we wouldn’t — couldn’t — have conceived a few generations ago. We can now learn so much from people all across the globe and share their experiences, dreams and knowledge. Real-time chat platforms, from IRC to FaceTime, have enabled us to gradually bridge the physical distances that separate us. Still, none are a substitute for real human interaction, face to face, in the real world.

Facebook Spaces, on the other hand, places two more barriers between us and our friends: the bulky VR headset and the digital avatar. Then there are the constraints that Facebook places upon its users. At the moment, Spaces users can’t express sadness or anger. The likelihood of this becoming the new normal for social interactions is about as likely as Disney’s recently shuttered Club Penguin becoming the new Facebook.

Facebook is asking us to take a chunk of our most precious commodity — our attention — and give it entirely to its virtual world. But that world is one step removed from that of the internet, full of (literally) fake people in a fake place. In the demo video, friends gather together a surprise birthday party to take a selfie of themselves on an imaginary boardwalk.

But where is the reality and authenticity in you, and three other people, taking an image of some cartoon avatars in a virtual world? Will you expect that future historians will unearth these images and marvel at the experiences that you pretended to have?

I wonder if I’m wrong, and that people will embrace this in the same way that they have taking pictures from inside video games. But then that, at least, shows a journey that you’re going on — and an appreciation of a particularly engaging sight that’s been created within the game. Imagine the reaction of your friends and family when you show them your vacation shots from the Grand Canyon that you didn’t visit.

Even Mark Zuckerberg knows that Facebook’s Social VR demo, as it is, not the future of how we’re going to interact with one another. Both this year and last, the CEO showed off a rough idea for an augmented reality platform that could work in a pair of glasses. There, you’d never need to buy a TV, just use the AR overlay in the lenses on conjure one up on the wall of your otherwise sparse living room.

Unfortunately, that idea is nowhere near ready for the real world, and Facebook VP Deb Liu told the AP that the journey to that hardware was “just one percent finished.” Imagine, rather than exchanging time-sucking pleasantries with a cartoon avatar while standing on a fake mountaintop, having someone sitting on your couch next to you.

You’d be able to speak to them as if they were in the room, despite being on the other side of the world. But with this (almost) real interaction, you would have real eye contact, real body language and all of the other things that we, as humans, need in order to feel comfortable interacting with one another.

The software and hardware just isn’t there for this sort of platform, and probably won’t be for a very long time. But that, not Facebook Spaces, is the vision of social VR that I can get behind.

Click here to catch up on the latest news from F8 2017!

* Technically not a selfie, since it’s not the person taking a photo.

From the stage of F8, Joaquin Quinonero, Facebook’s Director of Applied Machine Learning, described a new technique the company is using to improve the watching experience for 360 videos. The format is challenging to deliver because of its size, but Facebook is using machine learning to reduce the number of pixels that have to be rendered at any one time. By predicting where a viewer will look next, rendering priority can be given to that location  — particularly helpful for users with lower quality internet access.

The status quo for 360 videos is reactive rather than proactive rendering. Mike Coward, engineering director for Facebook’s VR video team echoed the frustration of users to me when he described the unpleasantness of turning your head in VR only to see a blurry scene.

One partial fix is to optimize compression. But teams at the company are already using machine learning to select across the thousand-plus compression techniques for individual snippets of video. The other way to reduce the streaming load is to just cut down on what you’re rendering. And rather than reduce quality across the board, Facebook’s approach improves resolution for exactly what you’re most likely to look at next.

Mike Coward, engineering director for Facebook’s VR video teamStep one was to use the resources of the company to monitor where people actually do look when watching 360 videos. Facebook’s VR video team created a heat-map that highlighted the most popular spots that users looked at within videos. From there, Facebook built a generative saliency map using a deep neural network. This model makes it possible to perform predictions on new videos that haven’t previously been watched or studied.

If a human were to be given the task of predicting where someone might look, they might study their natural environment and look for anomalies that could catch one’s interest — think birds or a car driving by.

Abstracting away to the neural net, the physical cars and birds cease to matter. Facebook’s model was trained on a massive corpus of videos to identify interesting subsets of a video frame. Coward told me that the model, when faced with a surfer in the ocean, is capable of picking selecting the surfer as most interesting, despite the fact that both are fast moving entities.

After implementing the prediction model, Facebook was able to increase resolution by 39 percent on VR devices. Aside from improving resolution and making 360 videos accessible to people without great network connections, the technology could some day make it possible to offer preemptive suggestions to creators on how to make videos more engaging.

Today Facebook open sourced Caffe2. The deep learning framework follows in the steps of the original Caffe, a project started at the University of California, Berkeley. Caffe2 offers developers greater flexibility for building high-performance products that deploy efficiently.

This isn’t the first time that Facebook has engaged with the Caffe community. Back in October, Facebook announced Caffe2Go, what effectively was a mobile CPU and GPU optimized version of Caffe2 (they even both have Caffe2 in their names if you parse it right). Caffe2Go received attention at that time because its release coincided with Style Transfer.

Notably, the company also released extensions to the original Caffe. The majority of these changes make Caffe more attractive to developers building services for large audiences. For projects where resources are of no consequence, Facebook has historically turned to Torch — a library it finds optimal for research use cases.

Every tech company wants to tout the scalability of its machine learning framework of choice. I asked Yangqing Jia, the lead author on Caffe2, what he thought of MXNet and the noise Amazon has been making about its ability to scale. Reasonably, he was cautious about dropping benchmarking numbers for comparison. These numbers can have meaning, but they are heavily influenced by the actual implementation of a machine learning model and subject to a fair amount of “DIY” volatility.

Yangqing Jia, the lead author on Caffe2 and Alex Yu, leader of business development

“All frameworks are more or less at a similar scalability factor,” explained Jia. “We’re pretty confident that Caffe2 is probably a little bit better than the rest.”

Facebook is pouring a lot of resources into both Caffe2 and PyTorch. Today’s release accompanies partnerships at the hardware, device and cloud levels. Alex Yu, leader of business development for Caffe2, explained to me that Facebook aimed to include the market leaders in each category. This meant Nvidia and Intel on the hardware side, Qualcomm on the device side and Amazon and Microsoft on the cloud side. And while Google wasn’t targeted, a GCP partnership wouldn’t be out of the question going forward.

Prior to release, Caffe2 was deployed at scale across Facebook. The team also took considerations for the developer communities familiar with the original Caffe. Caffe models can be easily converted to Caffe2 models with a utility script. Facebook is releasing documentation and tutorials and has put Caffe2’s source code on GitHub.

80% of Instagram’s users 600 million users are outside the US, so it needed a way to provide a better experience for users with limited network connectivity or no data plan.

Today at F8, Instagram announced it’s built support for using most of its features without Internet access. Much of this functionality is now available on Android, which is the preferred device type in the developing world. More will come in the following months, and Instagram tells me its exploring an iOS version.

Instagram engineer Hendri says offline users will be able to see content previously loaded in Instagram’s feed. People can leave comments, Like things, save media, or unfollow people — all of which will go through when they reconnect. Profiles they’ve visited before will be visible, as will old versions of the Explore tab or their own profile.

The engineering gymnastics required to do this could help Instagram grow in developing nations where data is either too expensive for everyone to afford, or there aren’t omnipresent or stable data connections. Facebook’s developing world app Facebook Lite shot to 200 million users in just a year, proving the big opportunity Instagram could seize by allowing users to enjoys the app even in isolation. While Snapchat seems to have forgotten about the developing world, Instagram knows everyone everywhere wants visual communication.

I had the chance to try out Spaces for myself a few hours after it was announced at F8. As soon as I placed the Oculus Rift headset on my head and the Touch controllers were in my hands, I was transported to what appeared like a beautiful park with cherry blossom trees. On my right was my virtual helper, Justin, who appeared in the form of an animated cartoon avatar. In front of me was a tableau of sorts, with a little dashboard in front of me.

Justin told me to select Appearance, and voila, I could customize the appearance of an animated cartoon avatar of myself. You could design one from scratch by customizing individual features like your nose or your hair, but I decided to just have one automatically generated for me. I grabbed one of my profile photos, which were already on display, and Spaces was smart enough to translate it into a cartoon version of me. From there, I also changed the color of my glasses and my shirt, which you can do in the Appearance tab too.

What I noticed almost immediately is how real it seemed, which is really weird considering I was speaking to an animated avatar. Mike Booth, the product manager leading the Spaces development team, says that’s because the Rift and the Touch creates a little motion capture studio. “You get your actual body language,” he said. “It captures head movements, even hand gesticulation.”

What’s more, Spaces also infers what your eyes are looking at, creating what appears to be eye contact, which is integral to face-to-face communications. One of the reasons Spaces can do this so well is because these avatars are stylized and cartoon-like. “They’re not hyper-realistic, where you can find every little flaw,” says Booth.

The same goes with mouth movement. It actually listens to the voice coming through the Rift microphone and then it tries to guess what mouth shapes you’re making. It’s not always accurate — it sort of snaps the mouth around like a Wallace & Gromit cartoon — but that’s entirely on purpose. “We’re not trying to be super photorealistic,” says Booth. “We just want to show that you’re talking.”

There are other fun things you can do with your avatar too. If you point both thumbsticks up, your avatar will laugh. Point them out, and you’ll smile. Place both controllers on your face and you’ll make an “OH” face. Turn them outward, and your avatar will shrug and look confused. Gestures are basically to VR as emojis are to text. “You have to invoke them,” said Booth. “They’re not supposed to be accidental.”

Next, Justin showed me how to share different kinds of media. I could share photos and videos from my own album, or I could share ones from my newsfeed, or I could just share whatever on the web I found interesting. Once selected, I could resize them anyway I want and have them displayed in the background. What’s especially neat is that when you share 360-degree photos and videos, you can sort of throw them in the middle of the tableau and the image will completely envelop the world. It sounds sort of silly perhaps, but when Justin shared a 360-degree video of a CNN documentary of Iceland, I almost felt like we were right there, taking a tour of the glaciers.

You also have the option of using a marker to draw silly doodles, turning them into three-dimensional art. You can toss them around, make duplicates of them or just play silly games with them. And of course, I couldn’t not take a selfie. Yes, there’s a virtual selfie stick, and yes it works just as it sounds. Simply grab the selfie stick, frame the shot you want and snap the perfect shot. From there, you can share it with your friends just like you can with any other photo.

Last but not least, Spaces is also tied in with Messenger. So you can initiate video calls right when you’re in VR. Your friend’s video chat screen will show up in a little floating square (No, they don’t need a VR headset to participate) and only you can hear and see what he or she says — none of your other VR buddies can eavesdrop in the conversation. They can still hear what you say of course, but they can’t hear what your Messenger friend is saying. Facebook tells me there’s no way to enable that just yet, but that functionality might come later in the future.

As cool as all of that is though, what really sold me the most on Spaces are those animated avatars. And they also happen to be one of the factors that was really hard for Facebook to get right. “The biggest challenge was how the avatars were going to look,” said Booth. “You know the uncanny valley? Well the uncanny valley in VR is a lot wider. Anything that’s attached to your head is going to have biological motions; it’s going to seem alive.” That means that anything that was too life-like would look weird.

“Finding the right balance of charming and being human recognizably without being too realistic and creepy, was harder than expected,” said Booth. “One of our early experiments was to have the lips smoothly blend with the shapes, thinking it would be realistic. And it was just way too creepy.” There were also some experiments on non-human avatars, like you’d see on Zootopia, but that was nixed as well. “Facebook is about authentic identity, which is fundamentally about humans.” Still, that doesn’t mean that costumes won’t be an option later on.

“The core of Spaces, the reason it exists, is so you can feel like you’re in person with your friends,” said Booth. “And then it’s having interesting things you can do with your friends. It’s not a chatroom where you’re just talking.

Of course, Facebook Spaces isn’t the only social VR app out there. Oculus even has its own version called Oculus Rooms. The difference between the two, Booth says, is that Oculus’ version is made just for Oculus hardware and is made to drive that particular platform. Facebook Spaces, on the other hand, is supposed to be much more widespread. That’s why even though it’s an Oculus exclusive right now, Booth wants it to be on all VR hardware. Yes, even the Vive.

When I asked what Booth would say to the skeptics of social VR, he said he doesn’t quite know. But what he thinks would really happen, is that VR skeptics would finally be persuaded to use VR because of social. “There are people that are looking at VR, thinking it’s not for me, because they think it’s all about gaming,” he said. “But what we’re trying to build is for everyone.”

“We want to bring you and your friends to VR,” said Booth. “I hope it’ll make more people look at VR as someone that people will actually want.”

Facebook Spaces aka Facebook In VR

If there was still any question why Facebook paid all of that money for Oculus, today’s F8 keynote provided some extra explanation. From AR to social VR and more, the company laid out its plan for the immediate future that involves blending the virtual world with real life. If you missed the hour-long talk earlier today, don’t fret: We’ve compiled all the big news in a 10-minute clip.

Click here to catch up on the latest news from F8 2017!

The full version of Workplace will be free until September 30th, but Facebook plans to charge a modest monthly rate of $3 per person for the first 1,000 people, $2 per person for the next 9,000 users, and $1 per person after that. Slack does offer 5GB of storage and service integrations with its free service, but you’re looking at a minimum of $6.50 per month for perks like additional storage or searching chat archives. That can add up quickly if you’re part of a large outfit.

Workplace is also getting bots that can automate tasks in group chats, such as ordering food for a long meeting or Lyft rides for the trip to an event. You’ll also have the option of broadcasting live video from pro equipment, such as a high-end camera at a presentation.

Should Slack be worried? That’s hard to say at this stage. Facebook’s name and resources will certainly get its foot in the door, but Slack has years of lead time and a presence on virtually every major platform. Also, some organizations may prefer Slack precisely because it isn’t Facebook. You don’t have to think about sharing your Facebook identity with your office, for example. It won’t be surprising if Workplace makes further inroads with its free tier, but it isn’t guaranteed to dominate.

Click here to catch up on the latest news from F8 2017!

Facebook teased how it would create social experiences in virtual reality, and now it’s ready to take things one step further. It just launched a Facebook Spaces beta that lets Oculus Rift owners hang out together (up to four at once) in VR. You can create an avatar that’s as realistic or fanciful as you want, draw 3D objects and show off your personal videos and use 360-degree video backgrounds to liven up your chats. There’s even a selfie stick to take virtual self-portraits. And if a friend can’t use VR, you can bring them into the conversation using Facebook Messenger video calls.

Developing…

Click here to catch up on the latest news from F8 2017!