Expelliarose follow-up (and Parrot Awards!)

Despite the end of ROSE, we’ve been working hard on the Expelliarose project the past 2 months. We knew we were close to a really nice product and we didn’t want our work to end on a mitigated result (those who attended the presentation or watched the demo know we had a few remaining bugs).

Therefore, we put a lot of work on solving the issues we identified:

  • IR protocol malfunctioned in some cases: the reception is now more robust (up to 5 meters, which could be improved with a higher power LED),
  • Some gestures were too close to be distinguished: we redesigned most of the gestures and crafted an extensive database with different people to account for the multiple ways to grasp the wand,
  • Some thresholds had to be adjusted: the DTW algorithm yields quite noisy results (well, our implementation does) that must be processed to be relevant. We are now able to accurately detect any of our 10 gestures while the “null gesture” (ie. an unknown movement) is identified most of the times (but occasionally gives a mismatch, still work to do on that one),
  • The Android application crashed from times to times: we broadly tested many scenarios to track and fix those bugs, and improved the UI to give the user more output about the ongoing battle,
  • The wand was quite large and looked massive: we completely redesigned it to make it look more appealing. We got rid of the screw thread, we changed all the board connectors to much smaller ones, we switched to slimmer (but lower-capacity) batteries so that the PCB became the only limiting factor, we changed the wand’s head to simplify its assembly, and we added some tasteful decoration (at least we believe so),
  • “Expelliarose” was incomprehensible to pretty much everyone but the ROSE crew: we decided to adopt the name Incanto, that sounds like a spell and evokes incantations, therefore magic.

We presented this refreshed wand to the Parrot Awards jury last week.
We attended the award ceremony yesterday, and we are super proud and happy to announce that we received the first prize of the competition!

Therefore, amazing news: Incanto will go to the CES 2016 in Las Vegas with the Parrot delegation!

Meanwhile, we’ll continue to work on the wand to make it even slimmer and prepare, hopefully, a crowd-funding campaign. Will you join the wizardness? Follow us on Twitter to get updates (yet, we might take a summer break first…).

We thank Parrot, Sam & Alexis for their trust and their help. To be continued!

Updates about the Android application

Since a picture speaks a thousand words, I’ll let you do the maths to find out how many blog posts will I save by showing you some screenshots of our first Android application!

This might not seem much, but we’re now able to play with multiple wands since the application assigns an unique identifier to each one. And that’s a great way to make sure spells are sent and received.

Screenshot_2015-04-14-20-06-51

Screenshot_2015-04-14-20-07-09

Screenshot_2015-04-14-20-07-28

Next step: the server!

(Note that this is a debug application, some options are not meant to be part of the final product)

Take that spell

After a few more days to tune the API between the nRF and the STM32, we’ve managed to get some interesting results. We’re now able to detect an incoming spell, to trigger wonderful LEDs animations, and most of all to broadcast it to the client. It is also working the other way around (if the server decides to attack the wizards!). The communication between the phone and our PCB is operational, and I have worked on the wand’s main thread to gather the different modules we’ve all been building. We have chosen to use an event-driven structure to keep the different parts of our code quite independent (that way, we can parallelize our efforts) and this choice seems so far efficient enough for our purpose. The connection between the BLE, the IR emitter/receiver, the LEDs and the motor is functional.

The rest of the team is working hard on the IMU, since movement recognition is a crucial part of our project. The last part of our 3D-printed wand was successfully printed today ; we made it very practical, and we will try to make it shinier when we have some time. We also decided to slightly switch priorities: instead of focusing on firmware flashing over USB (which is not essential for a prototype), we’ll try to set up the microphone when the human power becomes available. Yelling wizards would definitely improve the game play!
Oh, and we are struggling on the capacitive controller that yields unexpected (or un-understood) results. We’re not giving up since it would provide a much better user experience than a regular, poor, depressing switch. The great magi have been summoned. Continue reading Take that spell

A week in the BLE desert

Long time no post, because, well, I didn’t feel like bragging about my struggle with the Nordic chip.

But here we are. Let’s make the most of it! The mission (I accepted): building the communication between the STM32 and the user’s client (a bluetooth low energy device).

Quick reminder: we had decided to use a SPI bus to connect our STM32 and the Nordic nRF51822 chip. But since we only added one chip select wire, we realized our schematic didn’t fit our purpose: both chips should be able to act as master and could want to initiate a transaction. Hence the decision to go back to the UART (we’d planned this, but not thoroughly enough: the RTS and CTS pins are not mapped, so we’ll lose the hardware flow control).

Starting from there, and discovering with enthusiasm that the last Nordic SDK 8.0.0, freshly released, was compatible with our evaluation board (PCA10001, embedding the same chip than us), I developed a custom 8.0 BLE service (with the numerous Nordic examples as a base). Continue reading A week in the BLE desert

Exhausting challenge

Here’s the end (well… is it?) of a demanding week, in which we all worked on our TP, and today on the challenge (aka the evil and picky Scala Ada server). We probably did not spend enough time on the project those past few days but that was required in order to become familiar with ChibiOS, which is a great piece of work with a few documentation issues (but who could blame Giovanni for that).

So, to sum it up, we’re now able to:

  • use the threads and synchronization primitives,
  • use the network (lwip),
  • use the timers,
  • use the interruptions,
  • use the ADC,
  • use the GPIOs,
  • use the SPI driver,
  • use the serial over USB,
  • write valid code in one shot.

And most of all, we now start to picture the structure of a ChibiOS-based firmware.
Those basis give us great insights of the coming weeks, brace ourselves… and let’s get some sleep.

chibios is coming

Routing… routing…

Hi,

One more day routing and placing, this time straight after an Athens session in Brussels (hopefully the Thalys is fast). We’re working with a small team since some of us are abroad, and it is a bit difficult to reach a satisfying workflow with only one instance of Expedition PCB… It seems like we’re making baby steps, linking pins one at a time in a single session! Oh, we could so benefit from a distributed version…

As we had been warned, automatic routing yields very odd paths. Therefore we’re carefully drawing them by hand, with love (one could argue that the result is equally strange, but eh). Yet, the shape of our PCB make the task rather tedious… and we’ve only routed 54% 🙁 − a lot of the missing routes are VCC or GND, but we are definitely not there yet. Btw, what is the cleanest way to route those? Expedition PCB’s suggestions sometimes seem randomish…

Oh and we’ve managed to gain a few millimeters in width above the CPU in order to get a perfectly shaped wand (almost), so that’s an achievement. We’ll remove the receivers from the PCB in order to make it smaller (money money) and to give us more flexibility to construct the Expelliarose. We’ll probably have to use 4 of them since we’ve not been able to test the IR LEDs, simply because we can’t get a hold on them… someone from Slytherin might have hidden them, hopefully we’ll found them in a few days.. but as of now we can’t do much. Murphy etc. (or just bad karma).

Hugues and I will now get some rest in order to be fresh as mandragoras for our tomorrow classes. Might the wizards of routing be with us.

Sunday, day of the lord Voldemort

This pun is way too easy and I am already wasting it, not even a week after we started to work on Expelliarose… Hopefully you’ll forget that and we’ll be able to use it again later.

Today, we worked on our PCB, getting some inspiration from the LEDZep project and mostly from obscure datasheets. The BLE module, IMU, power modules, IR emitters/receiver, are all set, but we still need to route some pins from the STM32-F427, and wait for a few components to be added in Expedition PCB.

Speaking of components, we now have picked them all, completing our first two PSSC. We decided to go with an other IR receiver, high power LED, and touch controller, following Alexis’s advices. We’ll be using the remaining LEDs of the StabiloRose project, slightly more dense than the ones we were supposed to use, yet their protocol seems equally dirty. We’ll also have to make sure we’re not turning them on all at the same time to be gentle with our battery 🙂

Our workflow is getting better and we’re learning the hard way how to make sense of huge specifications. Since we’re going to make an expansive circuit board, we’re reading them over and over again to make sure we’re not going to miss the ideal pin for our usage. We don’t have much of a method to make routing choices, but that’s how we build experience!
We are also computing capacitors and adding all the fuzzy details around our main components, and we are definitely feeling like doing some voodoo. At least this is in the theme.

We’ll finish that tomorrow if the owls of the circuit boards lead us in the right path!

Need suggestions on (infrared?) targeting!

So, we did set up our Trello today, powerful tool but the “lists” don’t strike me as being the optimal way to organize our work, since they’re a bit messy. We’ll deal with them though.

Concerning our project, Expelliarose is under way, and we have to present our calendar and objectives on Friday. And design our PCB on Tuesday, waw.

Therefore, not a second to waste, let’s find out how our magic wands will target their opponents !

Short remainder of the use case: magicians will hold a magic wand and target their opponents. We shall detect the magician our user is pointing to in order to send him/her a spell.

Multiple alternatives here. Some don’t seem to fit our requirements:

  • very inaccurate targeting with BLE : our wizard sends a spell and every magician in range would take the hit… not so great in terms of gameplay (still could be fun for some specific spells),
  • targeting with a laser: probably too accurate, and most of all a signal quite hard to modulate,
  • ultrasounds: would we fit them in a wand thinner than a log…? Probably not, and it would require a mic constantly listening, not so much for energy consumption and saving the planet.

So, still in play :

  • High luminosity LEDs, infrared or visible light. Several challenges :
    • Range (we could maybe achieve 20m in low luminosity conditions, probably less if we want to play in the sunlight): is it gonna be enough?,
    • Precision (how can we make the signal directional enough to distinguish two wizards spaced by 2 meters at 15m? Lenses? What kind?),
    • Detection (we probably need at least 4 receptors to cover all directions, and it will require the two wands to be in sight ; cameras don’t seem to be an option (expansive and too large/heavy to be embedded at the end of the wand)). Plus, we also want to match the wands when they are facing each other, so we would probably also need a receptor at the end (next to the LED…). That’s tricky.
  • And… well, no other option comes to mind.

Do you have any thoughts? What should we do to ensure a robust (sort of…) targeting ? Any pitfall to avoid? Visible or infrared? Or another technology? Suggestions are more than welcome to provide a good UX to our players. Thank you!

Expelliarose en image

We’ll keep you posted shortly!

Inauguration

A not so thrilling day for a first logbook entry:

– The power board of my laptop seems to have died,

– 3 studious TH with Alexis,

– Flo and I decided how should we work on the BLE tutorial,

– First thoughts (in French) on the first project, sent to the mailing list (that will probably never publish them):

Avant de parler technique, voici une proposition de scénario de jeu (sans trop faire attention aux contraintes) pour qu’on parte sur les mêmes bases. Qu’est-ce que vous en pensez ? Ça peut être assez marrant.

– Les participants (sorciers) ont une baguette connectée à leur smartphone, qu’ils portent à la main ou à la ceinture.
– Un sorcier peut saisir sa baguette, la pointer vers un autre sorcier.
– La baguette s’active, il peut alors jeter un sort avec un geste bien précis, en récitant éventuellement une formule.
– La baguette y réagit (son, lumière, vibration, …), mais le sort n’est pas instantané (pour laisser du temps à l’autre, les avada kedavra dans le dos c’est lâche).
– En parallèle le smartphone de l’attaquant contacte le serveur et l’informe de l’attaque.
– Le serveur décide si : l’attaque est autorisée (niveaux un peu équivalents, que Dumbledore ne s’attaque pas à Neville), si le sort est autorisé (valide, bien fait, “débloqué” en fonction du “niveau” du sorcier, si le sorcier a suffisamment de points de mana pour l’envoyer).
– Si oui, le serveur contacte le smartphone de l’attaqué, il le notifie (vibration du téléphone ou de la baguette, son (formule), …)
– Si non, la baguette montre au sorcier que le sort a échoué.
– Si le sort est valide, l’attaqué doit se dépêcher de faire une parade (validée ou non par le serveur comme ci-dessus).
– Un combat peut ensuite avoir lieu si les sorts ne sont pas létaux (ils peuvent diminuer le karma, faire baisser une “jauge de vie”, …)
– On peut envisager plusieurs types de sorts de combat (la baguette tremble, le téléphone hurle et vibre, la baguette envoie une décharge, on devient indétectable/inattaquable) et même des sorts un peu plus longs (l’autre regagne plus doucement des points de karma, l’autre blesse ses coéquipiers quand il utilise un sort à leur proximité, etc., et leurs contraires).
– Le combat se termine quand le serveur décide un vainqueur (après un sort mortel, ou avec une heuristique de combat : celui qui fait le mieux les gestes, qui crie le plus fort, qui vise le mieux, …). Le bilan du combat a un impact sur l'”expérience” des sorciers (déterminé par le serveur).
– L’expérience des sorciers débloque des sorts, des parades, …
– Le tout peut se faire en équipe (règles classiques : dernier en vie, sorcier à protéger, …).
– Le serveur peut éventuellement “provoquer” des combats quand deux joueurs sont à proximité (en allumant les baguettes, comme l’épée de Frodon).
– … ?
La baguette pourrait aussi avoir une fonction d'”apprentissage” pour guider le geste des sorciers qui apprennent un nouveau sort.

Et ensuite, quelques réflexions sur le système :

On parle donc d’une super baguette magique (a priori imprimée en 3D ?) avec des capteurs (Alexis évoquait la centrale inertielle, vu le scénario ci-dessus on peut aussi penser à un micro pour entendre/confirmer le sort qui va avec le geste, un capteur de pression pour que la prise en main de la baguette joue également, ..), et des LED/retours haptiques pour donner vie à l’action en cours (pourquoi pas un mini haut-parleur pour seconder le téléphone, un mini buzzer électrique pour faire lâcher la baguette sur certains sorts, ..?).

Si on a plusieurs personnes en face de nous il faut aussi un moyen de viser (infrarouge par exemple ? pas très précis ni très robuste en extérieur.. vous avez des idées ?). À moins qu’on ne choisisse notre adversaire au préalable sur notre téléphone (mais niveau gameplay ça casse pas mal le truc).

Il faut ensuite que la baguette transmette ces infos au smartphone (connecté en Bluetooth Low Energy). À lui ou au serveur d’identifier le geste, le son, etc. Si un des objectifs est d’empêcher les gens de tricher il faudra réfléchir à authentifier le sort (donc la baguette et/ou le téléphone), ce qui n’est pas un détail puisque ça a un impact direct sur la construction de l’API.

Il faut ensuite reconnaitre la signature d’un geste à partir des données inertielles et de la signature vocale en la comparant à une BDD, c’est de l’info rigolote. On peut faire ce travail au niveau du téléphone pour minimiser les données échangées avec le serveur, ou niveau serveur pour, entre autre, être sûr qu’on utilise la même version de la BDD des deux côtés et restreindre la possibilité de frauder.

L’API reprendrait globalement les contours du scénario avec des objets Sorcier, Sort, Parade, …, qui pourraient interagir entre eux.

Le serveur fait ensuite le lien entre les smartphones (ou est-ce qu’on essaie de le court-circuiter un peu pour éviter la latence ? wifi direct, bluetooth (pas LE), …?). Le “ou pas” veut dire qu’il faut vérifier qu’on arrive à faire des aller-retours incessants avec le serveur sans détruire l’interactivité du jeu (si on a une seconde d’attente pour que le sort arrive dans la baguette du voisin ça risque d’être chiant). Le serveur permet de lancer une partie et d’en définir les règles, les joueurs peuvent rejoindre une partie avec un identifiant (ou en NFC, etc.). Il valide le déroulement de la partie, authentifie les joueurs et maintient leur profil à jour. Il se charge de l’expérience et des équipes et il peut par exemple envoyer de nouveaux sorts vers les téléphones des sorciers (qui sont notifiés) au fur et à mesure qu’ils sont débloqués.

– Subsequent email exchanges and sparse thoughts on the Battle Royale project,

– Translation of this blogpost in order to match the (too easily forgotten) instructions of one of our BDFL Alexis.