QueTwo's Blog

thouoghts on telecommunications, programming, education and technology

Tag Archives: Flash

Creating a one-time login for a mobile AIR Application

One of the aspects of Pointillism was that we wanted to minimize the amount of time that the user needed to worry about logins, passwords and signing up for the service.  This should be a pretty common goal for most mobile applications — the more you force the user to input that type of information into your app (or verify, re-verify, etc), the less chance they will use it.

We decided to base the app around the “ViewNavigatorApplication” model within Flex.  For the rest of the application, it made perfect sense as this type of app could easily be built around “screens” that were stacked as the user moved from one activity to another.  The problem was — if I wanted to force the user to login, I would either have to introduce some sort of “launching” screen that would contain the logic-check to see if the user had logged in prior, or I could not define the “firstView” property of the application tag and have some script in the Application tag decide.

My solution consisted of this — I defined the firstView to go right to the dashboard within the application (so, where a logged in user would go).  I then added a bit of code to the initialize event handler that could intercept the creation of the View and force it to go to the login screen ONLY IF the user had never logged in before.  This allowed the normal operation of launching the app after the user had logged in to go very quickly, yet still force the login in a seamless way.  This also meant that the user wasn’t subjected to multiple awkward transitions as the application decided if they were logged in or not.

<s:ViewNavigatorApplication xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark"
 initialize="preAppInit()" firstView="org.pointi.views.MainScreen" .....>

import ....
public function preAppInit():void
 var userInfo:CurrentUserManager = new CurrentUserManager();
if (!userInfo.isLoggedIn())
 navigator.defaultPushTransition = null;

You will note that I set the default “push” transition because I wanted it to seem that the application launched right to the login screen, instead of having it flip to the screen (giving the impression that the user could hit the back button to go back to another screen).  Otherwise, the rest should be pretty self-explanatory.

Simple Caching Techniques in Adobe AIR

One of the aspects of the Pointillism mobile app that was recently released was that users were expected to use the game while in remote areas.  Remote areas often mean that data service is limited or just plain not available at all, and that can wreck havoc for game participants waiting for data to load.  There are two schools of thought in how to approach this problem.

One is to pre-load all the content that the game would or could ever use.  This means that you either package all the data / images with your app, or you force the user to download this data when they launch the app.  The advantage of this method is that the user can pretty much be completely offline after that point and still get the entire experience of the game.  The disadvantage of this, of course is that then you front-load ALL of your content.  If the user is on EDGE (or worse!), this would mean they would be downloading a LOT more data than they may need to in addition to making your app use more space on the end devices.

The other method is to setup some sort of caching strategy.  This requires the user to be online at least for the initial exploration of each section of your app, but after that, the data is stored on their device.  This can be problemsome if they are offline, of course, but depending on the game, this may not be an issue.  In a cached mode, the user will attempt to read from disc and return that data WHILE making the call to the service in order to pull down the latest data.  To the end user, this becomes transparent.  Updating cached data is also routine as all if you have to do is invalidate the cache to get that bit of new data.

In Pointillism, we worry about two types of data — lists of data (Collections, Arrays, Vectors, etc.), and user-submitted images.  Our goal is to cache both.

Luckily, Caching the images was super easy.  Dan Florio (PolyGeek) wrote a component known as the ImageGate which houses an Image component and a caching mechanism.  Using his component is as simple as substituting the <s:Image> in your MXML or ActionScript with his component, and boom — your images are cached as soon as they are viewed.  I did make a few tweaks to his component and posted it on my space over at Apache.  I substituted the Image component with a BitmapImage for speed, and added a small patch to cache the images in the proper location on iOS devices.

Caching lists of stuff was not much harder.  AIR has a built-in “write to disc” functionality known as SharedObjects.  SharedObjects started as an alternative to cookies in the browser, but within AIR allow us to store variables for long-term storage.  In my case, I choose to store data that came back from the server as a SharedObject every time we got some data back.  This turned out to be a good strategy as it allowed us to show old data immediately  and update it with current data once it came in.  Our data didn’t change /that/ often, so it might update at most every day or so.

One of our data manager’s constructor looked like this :

so = SharedObject.getLocal("org.pointi.cache");
 if (so.data.pointsList == null)
 so.data.pointsList = new Array();

When we got our data back from our server, we did this :

so.data.pointsList[curHuntID] = event.result as ArrayCollection;

And finally, when we wanted to read back the data, this is all we had to do (pointsList is the variable that was sent to our calling components):

ro.getPointList(huntID, userID); //call the remote function on the server
if (so.data.pointsList[huntID] != null)
 pointsList = so.data.pointsList[huntID] as ArrayCollection;

Pretty simple, eh?  We did similar setups for all of our data lists, and also implemented some caching for outgoing data (like when the user successfully checked into a location), so we could keep the server in sync with the client.

Compiling the Apache Flex SDK with IntelliJ

I’ve only been using IntelliJ for a few weeks now, but I love it.  I see myself using this as my primary IDE for all things Apache Flex as time moves forward.

One question that has been asked quite frequently on the Apache Flex Dev mailing list is “How do I compile the Apache Flex SDK with IntelliJ?”  Well, since a picture is worth a thousand words, a video on the subject must be worth… umm.. (11 minute video, at 15 frames a second, times the value of pi… )  9,900,000 words!

Compiling Apache Flex SDK with IntellIJ

  1. Grab the Requirements :
    1. Java JDK 1.5, 1.6 or 1.7
    2. Adobe Open Source Flex SDK 4.6 (needed for the compiler at the time of writing)
    3. IntelliJ with ANT, Flex and Java plugins
  2. Create a new Project
  3. Create a new Java Module.  Name it anything you wish.
  4. Create a new Flex Module within that last Module.  It must be named “frameworks”
  5. Unzip the contents of the Open-Source Flex SDK into your Java Module EXCEPT the frameworks directory.
  6. Check the frameworks directory from the Apache SVN (https://svn.apache.org/repos/asf/incubator/flex/trunk) .  Make sure it ends up in the frameworks directory.
  7. Load up the ANT tab, and add the /frameworks/build_framework.xml  file.
  8. Hit the “Run” icon to start the compile.
  9. Drink a beer, or take a shower — depending on what the clock says.

After about 7 minutes or so (my computer compiles it all in 422 seconds on average), you should have a successful build, and a custom-compiled SDK!

NOTE:  The reason why we created two modules is so that you can create your own branch (or switch to somebody else’s branch) without having a whole lot of heart-ache.  All you would need to do is go to the framework module and change the branch you are checking out from.  This will allow you to create patches and submit them into JIRA against the current “patches” branch, instead of the trunk.

Using AIR Native Extensions for Desktop and Mobile

During Wednesday’s meeting of the Michigan ActionScript User Group, we covered what AIR Native Extensions are, where to find the best ones, and how to actually use them.  Includes demos from both Desktop AIR and Mobile AIR projects.

The two locations to find some of the more popular ANEs are :

Creating a Windows AIR Native Extension with Eclipse – Part 4

In this final of my 4-part video series, I show you how to import and use the ANE that we created in the last three videos.  We will be using Adobe Flash Builder 4.6 to import the ANE, and we will build a very quick sample application that will use the getTestString and getHelloWorld functions that we wrote in our native DLL written in C.

If you want a copy of all the final projects, you can download them here.  The ZIP file includes the CDT project, the compiled DLL, the ActionScript project, the compiled ANE and the project created in this fourth video.  Enjoy!

Working with AIR Native Extensions on the Mac

As I said in my previous blog post, creating the ArduinoConnector ANE for the MacOS X platform was frustrating.  It turns out that Apple ships a slightly modified version of GCC that is close to what is offered on the Linux/Unix and Windows platforms, but not 100% the same.  When I approached the project, I thought it would be a cake-walk — simply take CDT (or gcc command-line), and point it to my POSIX C code, and I should be a few clicks away from creating the .framework file needed for AIR.

Turns out that the only way to make a .framework file is by using Apple’s Xcode (unless I’m really missing something in the gcc man pages).  I wasn’t really in the mood to learn yet another coding IDE, but I did want to post some of the gotcha’s I found in creating the Mac version of the ANE.  These are not step-by-steps, but rather things that tripped me up.

  • When you create a new Xcode project, you must choose a Cocoa Framework project (under Framework & Library).
  • Add the “Adobe AIR.framework” by clicking on the project name, and then going to Add, Existing Frameworks.  The Adobe AIR Framework file is found in your “Flash Builder App Dir -> sdks -> 4.6.0 -> runtimes -> air -> mac”  folder.
  • The Adobe AIR Framework file auto-includes the header file, and the external library that you need to link to.  Don’t go searching for another header file, you don’t need it.
  • Create/Import your .c and .h files and attempt to compile before setting your project settings.  If you don’t try to compile before you set your project files, you won’t see 3/4 of the compiler settings you need to make things right.  This one threw me into a loop for at least a day.
  • All of the examples have you including the <Adobe AIR/Adobe AIR.h> file in your project — I found this works, but all it is doing is including the FlashRuntimeExtensions.h that exists within the Adobe AIR framework.  I decided to resist change and include the Adobe AIR.h file.  It works.
  • There are two places you need to set your compiler and project settings.  One is on the project itself (the top-most node in the Groups & Files tree), and the other is in the Targets -> ProjectName.framework node.  Set all settings in both places (they don’t cascade once it has been compiled once, which you have to do to get the GCC settings to show up right).
  • Make sure your project is compiling 32-bit frameworks, targeted to the 10.5 SDK.   I never found that documented anywhere, and it was sheer luck I tried those combos.
  • Make sure that you turn the optimization level to None (0).  If you don’t do this, things will seem to work, but if you use any of the FREGet or FRENew functions, they will produce seemingly random numbers (I was convinced it was passing me back pointers casted as integers, but who knows).
  • Make the following target and project settings changes that deviate from the defaults :
    • Architetures : 34-bit Universal
    • Base SDK : Mac OS X 10.5
    • Build Active Architecture Only – Checked
    • C/C++ Compiler Version : GCC 4.2
    • Linking Compatibility Version : 1
    • Linking Current Library Version : 1
    • Other Linker Flags : -flat_namespace -weak_Framework “Adobe AIR”
    • Runpath Search Paths :
      • @executable_path/../runtimes/air/mac
      • @executable_path/../Frameworks
      • /Library/Frameworks
    • Packaging Framework Version: A
    • Wrapper Extension : framework
    • Framework Search Paths :
      • /Applications/Adobe\ Flash\ Builder\ 4.6/sdks/4.6.0/runtimes/air/mac
    • Library Search Paths: /Library/Frameworks/Adobe\ AIR.framework/Versions/1.0
    • GCC 4.2, Generate Position-Dependent Code: Unchecked
    • Optimization Level : None [-O0]
    • Other C Flags : -fvisibility=hidden
    • Other C++ Flags:  -fvisibility=hidden
    • Precompile Prefix Header : checked
  • In order to export the proper function names for your initialization and finalization functions, prefix them with __attribute__((visibility(“default”)))

Once you are creating your .framework properly, you should be all set.  If you created truly POSIX or standard C code, there should be very little changes you should need to make your project work.

I do have to make one additional note about a bug people have been finding with Flash Builder 4.6, as it was released on 11/30.  If you are attempting to include an ANE into a project, the project will compile, but if you debug the project, it will complain that it can’t find the method name (not the initilizer or finalizer, but one of your other method names).  If you debug your project from ANT or the command line, it will work fine.  The work-around for this is the following steps :

  • Make sure you have ADL in your search path.  Edit the /etc/paths file and add a new line with /Applications/Adobe Flash Builder 4.6/sdks/4.6.0/bin
  • Create a new folder on your desktop, Document folder, or somewhere where you will find it.  Copy your .ANE into it.
  • Create a new folder within this folder, and name it MyTestANE.ane    (make sure to include the .ane — that is very important)
  • cd into that new .ane directory, and execute the following command :    unzip ..\ArduinoConnector.ane      (assuming you named your .ane ArduinoConnector.ane)
  • Go into the bin-debug folder of your project, and run the following command :    adl -profile extendedDesktop -extdir ~/Documents/Extensions/ Main-app.xml    assuming that the directory you created in the second step was named Extensions and was located in your Documents folder.
  • You can repeat the last step as often as you need with no additional work, assuming you don’t need to update your .ane.   If you need to attach the debugger / profiler , you can follow the steps outlined on this blog (but use about:blank instead of http://localhost)

Well, I think that is about it.  I hope I can help alleviate some of the frustration that I suffered for somebody else in working with all this :)

Connecting your Arduino to AIR using an AIR Native Extension

One of the things that has been bugging the the most when working with Arduinos and AIR is the fact that you needed some sort of proxy application running in order to allow the AIR (or Flash) application to talk to the Arduino.  You see, the Arduino only exposes a Serial port (via USB), and AIR is limited in this regard.  The way people have gotten around this so far is to have AIR create a TCP socket connection to some sort of Proxy application, which in turn opens the Serial port to the Arduino.  It works, and it works very well, however this is a separate application you have to ship, have the user configure, and run.  Heck, even me as the developer found this solution clumsy when demoing these solutions.

I finally decided to fix the problem and write an AIR Native Extension.  The AIR Native Extension (ANE) is a C based .DLL / .framework for the Windows/Mac platforms that allows AIR to essentially open a COM port.  I wrote it in a way that is supposed to emulate the functions of the flash.net.Socket library that is included in the AIR runtime.  I’ve posted the entire project, including the source code and final binaries on Google Code at http://code.google.com/p/as3-arduino-connector/ (well, everything except for my compilation scripts, which are specific to my computers). 

The biggest learning experience in creating this ANE was developing on the MacOS platform.  I’ve never done any programming targeted for that platform before, and working with XCode is just a pain.  I’m used to environments like Visual Studio and Eclipse, but XCode always seemed to fight me every step of the way.  From simple things like hiding all the project properties, to trying to force you into an MDI workflow.  Also, working with the lack of documentation on the AIR Runtime side was kind of depressing…  Don’t worry, a future blog post will try to fill everybody in on how to make an ANE using XCode.

Using the ANE :

  • Include the ANE into your project.  Make sure you are using Flash Builder 4.6 or later. Right-Click on your project in the Package Explorer (Project Explorer), and go to Properties.  Go to the Flex Build Path tab, and then the Native Extensions tab.  Click “Add ANE…” and bring it in.  It does not need to reside within your project source folder.
  • Next, import the com.quetwo.Arduino.ArduinoConnector and the com.quetwo.Arduino.ArduinoConnectorEvent packages.
  • Instantiate a new variable of type ArduinoConnector.
  • Check the isSupported() function to make sure that the ANE is supported on your specific platform, and if it loaded properly.
  • Call the getComPorts() function which will return an array of valid list of COM ports on your OS.  On Windows, this returns ALL COM ports that are valid in Windows, where on the MacOS platform, it will return any USB Serial Devices, which usually would only be your Arduino.
  • Make the connection to the Arduino by calling the connect(comPort, baud) function.  You will need to pass in one of the COM ports from the getComPorts() array, along with the baud rate that your Arduino is operating on.  If you are using Firmata, it is 57600.  Most other Arduino projects use 9600.
  • Next, add an event listener to listen for the “socketData” event.  This will fire when new data is available in the data buffer.  Don’t wait too long to pull data out of the buffer, because it is only 4k :)

Sending Data:

Sending data is just like the Socket class, except you don’t have to flush() the buffer — the data goes out in realtime.  The two most common ways to send data :

  • writeString(“hello world”);
  • writeByte(255);

Getting Data from the Buffer:

As data arrives and is placed into the buffer, the bytesAvailable variable is incremented to reflect how many bytes are available.  Most people will read data from the buffer using one of the two functions :

  • readBytesAsString();
  • readByte();

All of the read functions are FIFO, meaning they will return the oldest data in the buffer before they return the newest (First In, First Out).

On the Google Code site, I am posting more detailed documentation, including a simple patch to as3Glue which will allow as3Glue to work seamlessly with this ANE. 

Enjoy, and please provide feedback on this ANE.  It’s the first one I’ve released to the public and I’d like to know how it works for everybody!

Creating a Windows AIR Native Extension with Eclipse – Part 3

In part 3 of this video series, I talk about how to write the AS3 portion of the AIR Native Extension, and how to package all the components into your final .ANE file. 

You can watch part one here, and part two here.  Part four will talk about how to use the .ANE within Flash Builder.  I will post all the files used in this tutorial in the final blog post.

My Presentations at Adobe MAX 2011

This year I had the distinct honor of being asked to present at Adobe’s MAX conference.  The conference was an absolute blast.  From the Keynotes to ALL the other sessions I attended, the thing went off without a hitch.

I gave two presentations this year — one for the 360|MAX Unconference and one for the Develop Track at MAX. 

Getting Data From Here To There (with Flex)

This session was not recorded, but fairly well attended.  You can see my slides here.   In this session I talked about the different communications methods available to Flex developers, and I started to lay out a basic matrix of when to use what type of communication method, and what the pros and cons were of each type.  Not all of my demos worked due to a broken J2EE server, but I think everybody got the idea.  I don’t have great downloads for that presentation as most were specific to my server setups.

Getting Physical With Flash (Hardware Hacking)

This session was a blast to present.  We had about 140 people in the room who seemed to be really into it.  I presented on integrating the Arduino hardware prototyping kit into Flash/Flex in addition to showing how to integrate the Microsoft Kinect into Flash/Flex. I came armed with about 6 electronics projects that I showed people to inspire them to create their own.

Video Player - MAX 2011 Preso

You can download the PPT here.

You can find most of the downloads featured on my blog, but I will update this post and post the direct links to everything at a later date.

Thanks again to the entire MAX staff for making the show run so smoothly from both the speaker’s perspective and from the attendee’s perspective.  A+ work led to an A+ experience :)

Integrating the Microsoft Kinect into your Flex application

One of the topics I will be talking about during my MAX session this year is integrating game controllers into your Flex/AIR or ActionScript games in order to make more naturally interactive games.  There are a lot of examples of using the Wii controller online already — and plus with the next generation of video game consoles coming out soon, I thought it would be best to spend my time on the latest and greatest — the Microsoft Kinect.

For those who don’t know much about the Kinect, it is a pair of Web Cams — one RGB (like a normal Web Cam), and one that sees only infrared.  The combination of the two (along with a bunch of other electronics and software) allow the Kinect to have real depth-preception.  This means that being able to track people, hands and everything else become trivial.  The Microsoft Kinect was actually developed by PrimeSense, who originally made the device for PCs, rather than game consoles.  This means that the devices are actually compatible with PCs, rather than relying on a whole stack of hacks to get them to work. 

The biggest difficulty in getting the Kinect working is wading through all the pieces and parts that are needed to get it all working.  Documentation is in very short supply (and usually found within text files on GitHub).  Compounding the problem is that there are tens, if not hundreds of open-source projects revolving the Kinect — but unfortunately each depend on a different set of drivers, middleware and APIs — many which are NOT compatible with eachother. 

I will state right here that I am not an expert in all of these, but I did follow one path that made things work (after following MANY that didn’t).  Matt LeGrand and I are planning a separate website and blog that will explore these other paths more to help people make sense of what to choose and what not to choose.  My primary experience is getting the Kinect working with Windows 7, as I am well aware that my aging Mac laptop is not powerful enough to do much with the Kinect.  Yes, you need a modern computer in order to use the Kinect.

So, where would you start?  You obviously need the actual Microsoft Kinect controller.  There are two models in the market right now — one that is USB only, and one with a USB + Power supply.  The USB only version will ONLY work with the XBox, where the USB + Power Supply version works with PCs (the Kinect draws a few amp of power, which the XBox apparently can supply using USB).  The only difference between the two packages is the power brick and a USB power injection module.  If you end up with the wrong one, you are supposed to be able to buy the power brick and injection module, but I have no idea where one would pick that up.  I got my kit from Best Buy in the video game refurbished aisle (it was about $100, after an instant coupon).  The brand-new ones were about $140.

Before you plug it in, you need to install the drivers, middleware and APIs.  There are three well-known sets of drivers and middleware, but the ones I found worked for me are directly from PrimeSence.  The drivers and middleware are hosted on OpenNI’s website at http://www.openni.org/downloadfiles/opennimodules.  There are three downloads you need — the OpenNI Module (OpenNI Binaries), the PrimeSense NITE Middleware, and the PrimeSense Drivers (OpenNI Compliant Hardware Binaries).  Download only the 32-bit downloads, even if you are on a 64-bit Windows 7 install!!  I’ve found LOTS of issues with the 64-bit drivers that cause things to break.    Install the drivers, then the middleware, then the OpenNI APIs (in that order).

Finally, you will need the as3 modules.  There is an open-source project known as AS3OpenNI that makes programming towards the OpenNI APIs very simple.  Because AIR can’t directly talk to the APIs you have to use the included C++ application that proxies the driver calls to a TCP/IP connection.  I’m sure this will become easier in future versions of AIR.   AS3OpenNI takes all the hard word of processing the blob data that comes back from OpenNI and gives you either skeletal data (as a class), RGB, Depth, or “multitouch blob” data.  With this data you can read back data from multiple users and track their hands, head, neck, legs, etc.  I built all of my stuff on version 1.3.0, which is really stable.

Take a look at the examples in the AS3OpenNI project — they are pretty descriptive, and once you get all the parts working together, they work really, really well :) 

So, what did I make using the Kinect?  One of my first games I threw together was a simple version of Space Invaders.  I have AS3OpenNI track the right hand to move the player left and right, and when the right hand moves over the center point of the neck, I fire a missile.  The following code is all that is really needed (minus the setup of the player, etc.) :

protected function gotShipMove(event:ONISkeletonEvent):void
 var rightHand3D:NiPoint3D = event.rightHand; // get the right hand's x,y,z
 var rightHand:NiPoint2D = NiPoint3DUtil.convertRealWorldToScreen(rightHand3D, this.stage.width, this.stage.height);
 shipIcon.x = rightHand.pointX;
 if ((event.skeleton.rightHand.pointY > event.skeleton.neck.pointY) && canFire)
  if (!fireInProgress)
   fireInProgress = true;   // prevent 5,000 missiles from firing at once...
   var missile:MissileItem = new MissileItem(spaceInvadersCluster);
   missile.x = rightHand.pointX;
   missile.y = height - 64;
   missile.addEventListener("MissileFireComplete", missileFireComplete);
   missile.addEventListener("MissileFireHit", missileFireComplete);
  fireInProgress = false;

Watch my twitter stream for more information about our (Matt and I) new website dedicated to the Kinect and AS3 :)  I will also cover the game a lot more during MAX, so make sure to come!


Get every new post delivered to your Inbox.

Join 27 other followers