QueTwo's Blog

thouoghts on telecommunications, programming, education and technology

Tag Archives: AIR

Creating a one-time login for a mobile AIR Application

One of the aspects of Pointillism was that we wanted to minimize the amount of time that the user needed to worry about logins, passwords and signing up for the service.  This should be a pretty common goal for most mobile applications — the more you force the user to input that type of information into your app (or verify, re-verify, etc), the less chance they will use it.

We decided to base the app around the “ViewNavigatorApplication” model within Flex.  For the rest of the application, it made perfect sense as this type of app could easily be built around “screens” that were stacked as the user moved from one activity to another.  The problem was — if I wanted to force the user to login, I would either have to introduce some sort of “launching” screen that would contain the logic-check to see if the user had logged in prior, or I could not define the “firstView” property of the application tag and have some script in the Application tag decide.

My solution consisted of this — I defined the firstView to go right to the dashboard within the application (so, where a logged in user would go).  I then added a bit of code to the initialize event handler that could intercept the creation of the View and force it to go to the login screen ONLY IF the user had never logged in before.  This allowed the normal operation of launching the app after the user had logged in to go very quickly, yet still force the login in a seamless way.  This also meant that the user wasn’t subjected to multiple awkward transitions as the application decided if they were logged in or not.

<s:ViewNavigatorApplication xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark"
 initialize="preAppInit()" firstView="org.pointi.views.MainScreen" .....>

 <fx:Script>
 <![CDATA[
import ....
public function preAppInit():void
 {
 var userInfo:CurrentUserManager = new CurrentUserManager();
if (!userInfo.isLoggedIn())
 {
 navigator.defaultPushTransition = null;
 navigator.replaceView(LoginScreen);
 }
 }

You will note that I set the default “push” transition because I wanted it to seem that the application launched right to the login screen, instead of having it flip to the screen (giving the impression that the user could hit the back button to go back to another screen).  Otherwise, the rest should be pretty self-explanatory.

Simple Caching Techniques in Adobe AIR

One of the aspects of the Pointillism mobile app that was recently released was that users were expected to use the game while in remote areas.  Remote areas often mean that data service is limited or just plain not available at all, and that can wreck havoc for game participants waiting for data to load.  There are two schools of thought in how to approach this problem.

One is to pre-load all the content that the game would or could ever use.  This means that you either package all the data / images with your app, or you force the user to download this data when they launch the app.  The advantage of this method is that the user can pretty much be completely offline after that point and still get the entire experience of the game.  The disadvantage of this, of course is that then you front-load ALL of your content.  If the user is on EDGE (or worse!), this would mean they would be downloading a LOT more data than they may need to in addition to making your app use more space on the end devices.

The other method is to setup some sort of caching strategy.  This requires the user to be online at least for the initial exploration of each section of your app, but after that, the data is stored on their device.  This can be problemsome if they are offline, of course, but depending on the game, this may not be an issue.  In a cached mode, the user will attempt to read from disc and return that data WHILE making the call to the service in order to pull down the latest data.  To the end user, this becomes transparent.  Updating cached data is also routine as all if you have to do is invalidate the cache to get that bit of new data.

In Pointillism, we worry about two types of data — lists of data (Collections, Arrays, Vectors, etc.), and user-submitted images.  Our goal is to cache both.

Luckily, Caching the images was super easy.  Dan Florio (PolyGeek) wrote a component known as the ImageGate which houses an Image component and a caching mechanism.  Using his component is as simple as substituting the <s:Image> in your MXML or ActionScript with his component, and boom — your images are cached as soon as they are viewed.  I did make a few tweaks to his component and posted it on my space over at Apache.  I substituted the Image component with a BitmapImage for speed, and added a small patch to cache the images in the proper location on iOS devices.

Caching lists of stuff was not much harder.  AIR has a built-in “write to disc” functionality known as SharedObjects.  SharedObjects started as an alternative to cookies in the browser, but within AIR allow us to store variables for long-term storage.  In my case, I choose to store data that came back from the server as a SharedObject every time we got some data back.  This turned out to be a good strategy as it allowed us to show old data immediately  and update it with current data once it came in.  Our data didn’t change /that/ often, so it might update at most every day or so.

One of our data manager’s constructor looked like this :

so = SharedObject.getLocal("org.pointi.cache");
 if (so.data.pointsList == null)
 {
 so.data.pointsList = new Array();
 so.flush();
 }

When we got our data back from our server, we did this :

so.data.pointsList[curHuntID] = event.result as ArrayCollection;
 so.flush();

And finally, when we wanted to read back the data, this is all we had to do (pointsList is the variable that was sent to our calling components):

ro.getPointList(huntID, userID); //call the remote function on the server
if (so.data.pointsList[huntID] != null)
 {
 pointsList = so.data.pointsList[huntID] as ArrayCollection;
 }

Pretty simple, eh?  We did similar setups for all of our data lists, and also implemented some caching for outgoing data (like when the user successfully checked into a location), so we could keep the server in sync with the client.

Adding a GPS-driven map to your Adobe AIR app

Over the next few blog posts I’m going to be writing about some of the cool little features I implemented in a recently released app I worked on — Pointillism.  It is pretty rare that I can talk about an app I’ve released, but the verbiage in this contract allows me to :)

On the admin interface of the app, the customer wanted to be able to add a “point” to the game.  A point is a destination that the end user is looking for in this virtual scavenger hunt.  In order to have the admins be able to visually see what their GPS was returning, we wanted to map the location, as well as the bounding area that they wanted people to be able to check in to.  While our admin interface was pretty basic, the functionality had to be there :

GPS and Map solution on iOS and Android

While most people would instantly reach for Google Maps, we decided to use ESRI’s mapping solution.  They offer a very accurate mapping solution that is consistent across all the platforms in addition to being very flexible   The one thing that Google Maps had a hard time providing us was the ability to draw the fence in a dynamic manner, built with realtime data that came from within our app.  It was important for us to be able to see the current location, and the valid locations where people could check into for that point.  The hardest thing was having the ESRI servers draw the circle (known as a buffer).  ESRI’s mapping platform is available for use FOR FREE, with very limited exceptions.  As a bonus, they have an entire SWC and already pre-built for Flex/AIR.

So, how was it done?  It was actually pretty simple :

    1. Add the SWC from ESRI’s website to your project.
    2. Add their mapping components to your MXML file.  We added the mapping layer and then a graphic layer (where the circle is drawn).  The mapping layer, we pointed to ESRI’s public mapping service.
      <esri:Map id="locMap" left="10" right="10" top="10" bottom="150" level="3" zoomSliderVisible="false"
       logoVisible="false" scaleBarVisible="false" mapNavigationEnabled="false">
       <esri:ArcGISTiledMapServiceLayer
       url="http://server.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer"/>
       <esri:GraphicsLayer id="trackingLayer"/>
       </esri:Map>
    3. We added a few components to the MXML’s declaration section.  This included the defination of the “symbol” (the circle itself), and the Geometry Service (the thing that figured out how to draw the circle in the correct place).
       <fx:Declarations>
       <esri:SimpleFillSymbol id="sfs" color="0xFF0000" alpha="0.5">
       <esri:SimpleLineSymbol color="0x000000"/>
       </esri:SimpleFillSymbol>
       <esri:GeometryService id="myGeometryService"
       url="http://tasks.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer"/>
       </fx:Declarations>
    4. Next, we had to write some code to update the map and draw the circle in the correct place.  This involves a few steps, including taking the GPS coordinates from our GPS device, and creating a new “MapPoint” which holds those coordinates.  A MapPoint is exactly that, a single point on the map.  The thing about ESRI’s service is that it knows a LOT of different map coordinate systems — so you need to make sure you choose one that makes sense.  In our case, our GPS is returning back data in WebMercator format (lat/lon) other known as Spatial Reference number 4326, so that is what we are going to use to project that point to center our map.  Finally, we will ask the Geometry Service to return a “buffer” of a series of points that represents a circle x feet around the center of our map.   When the buffer is returned from the web service, we draw it using our graphic we setup earlier and push it to Graphics Layer that is sitting on top of our map.  This all happens in a matter of seconds.
      protected function gotGPS(event:GeolocationEvent):void
       {
       var mp:MapPoint = new WebMercatorMapPoint(event.longitude, event.latitude);
      updateMapWithFence(mp);
       locMap.scale = 4000; //this is a magic number for the zoom level I wanted.
       locMap.centerAt(mp);
       lastPoint = mp;
       }
      protected function updateMapWithFence(mp:MapPoint):void
       {
       var bufferParameters:BufferParameters = new BufferParameters();
       bufferParameters.geometries = [ mp ];
       bufferParameters.distances = [ checkinDistance.value ];
      bufferParameters.unit = GeometryService.UNIT_FOOT;
       bufferParameters.bufferSpatialReference = new SpatialReference(4326);
       bufferParameters.outSpatialReference = locMap.spatialReference;
      myGeometryService.addEventListener(GeometryServiceEvent.BUFFER_COMPLETE, bufferCompleteHandler);
       myGeometryService.buffer(bufferParameters);
       }
      private function bufferCompleteHandler(event:GeometryServiceEvent):void
       {
       trackingLayer.clear();
       myGeometryService.removeEventListener(GeometryServiceEvent.BUFFER_COMPLETE, bufferCompleteHandler);
       for each (var geometry:Polygon in event.result)
       {
       var graphic:Graphic = new Graphic();
       graphic.geometry = geometry;
       graphic.symbol = sfs;
       trackingLayer.add(graphic);
       }
       }

And that is about it!  cross-platform mapping made pretty easy.  The cool thing about ESRI’s mapping solution is the power behind it.  They offer things such as the typical driving directions all the way though “How far can a user see if they stand on the ground at this point”.   Since the components are native to your AIR app, they are fast and behave like you expect it to, without the mess of having an HTML overlay in your app.

Geolocation issues with AIR on iOS (with a fix!)

The recent “GM” version of iOS 6 (this includes the production version that is now shipping on the iPhone 5) broke one of the APIs that Adobe AIR depends on for Geolocation.

If you have an app that was published with Adobe AIR 2.7 through AIR 3.4, your app will either get rejected if you are just submitting it to the store, or it may be broken if it is already in the store.  This is in addition to some other feature interaction issues that have also cropped up in iOS 6.

The bug is that the Geo Location (GPS) subsystem is never actually called — it fails silently, just as if the user denied the application to use the GPS.

Luckily, there is a work-around, but it does require a re-compile of your application.

  1. Make sure your app is running at least AIR 3.0.   AIR 3.4 is currently in production and it works great with this work-around.  AIR 3.5 is on the roadmap, and addresses some other iOS 6 issues (including the ability to address the full screen).
  2. Update the <name> tag within your descriptor XML document.  Wrap the actual name of your application in an extra <text: xml:lang=”en”> tag.  For example,
    <name>
     <text xml:lang="en">MyGPSApp</text>
    </name>
  3. Update the version number.
  4. Make sure your app is compatible with the new storage requirements in 2.23.  Details are available here : http://blogs.adobe.com/airodynamics/2012/03/05/app-compliance-with-apple-data-storage-guidelines/
  5. Compile, and wait.  Your app’s GPS should now work in iOS 3, 4 and 5.

Hope that helps some people out!  I know it was a big sticking point when I had some apps recently rejected because of this incompatibility.

Microsoft Kinect and Adobe AIR

This past weekend I had the pleasure of working with the Michigan Historical Museum for their celebration of Michigan’s 175th Birthday.  I was asked to do a display involving the Microsoft Kinect, to showcase some of the possibilities of the technology.  I introduced the patrons to demos of my Kinect Space Invaders game in addition to the “dancing stick figures” demo.  Both demos were a huge hit, with the dancing stick figures drawing people in and the space invaders game showcasing a no-touch game that is highly interactive.

For both demos, I used the new as3NUI AIR Native Extension that is available here : http://www.as3nui.com/   Unlike my other Kinect projects I’ve worked with, this is the first that has taken advantage of the Microsoft Kinect SDK.  It was a huge relief to find out how easy it was for Microsoft’s SDK to install (the only thing that tripped me up was the .NET Runtime version that I didn’t have installed).  The ANE plugged right into it and fired up without issues, which was a huge relief considering how much of a pain in the rear that the PrimeSense NUI tooling to get setup.

Microsoft has finally released their 1.0 version of the Kinect for Windows SDK as of today (Feb 1st).  You can find out more about their SDK here : http://www.microsoft.com/en-us/kinectforwindows/.  Unlike the earlier, beta SDKs, they are finally allowing the Kinect to be used for commercial applications HOWEVER, you will need to buy and use one of their non-subsidized Kinect units for about $250.  No word yet if they will have re-furbished packaging like they do for the XBOX 360 Kinect units.  Developers can still use the XBOX Kinect for the time being, but Microsoft is highly encouraging for us to purchase the Kinect for Windows units vs. the XBOX ones.  It is still be be seen how driver compatibility is between the two units.

If you have any interest in checking out the Kinect Space Invaders game that I demoed this past weekend, you can install the Microsoft Kinect SDK (this was tested with Beta 2), plug in your Kinect, and install the demo from here.    I will eventually be publishing an updated version tested with the final SDK, but I will be waiting for the updated AS3NUI ANE that is being built to support the latest features of the SDK.

Creating a Windows AIR Native Extension with Eclipse – Part 4

In this final of my 4-part video series, I show you how to import and use the ANE that we created in the last three videos.  We will be using Adobe Flash Builder 4.6 to import the ANE, and we will build a very quick sample application that will use the getTestString and getHelloWorld functions that we wrote in our native DLL written in C.

If you want a copy of all the final projects, you can download them here.  The ZIP file includes the CDT project, the compiled DLL, the ActionScript project, the compiled ANE and the project created in this fourth video.  Enjoy!

Working with AIR Native Extensions on the Mac

As I said in my previous blog post, creating the ArduinoConnector ANE for the MacOS X platform was frustrating.  It turns out that Apple ships a slightly modified version of GCC that is close to what is offered on the Linux/Unix and Windows platforms, but not 100% the same.  When I approached the project, I thought it would be a cake-walk — simply take CDT (or gcc command-line), and point it to my POSIX C code, and I should be a few clicks away from creating the .framework file needed for AIR.

Turns out that the only way to make a .framework file is by using Apple’s Xcode (unless I’m really missing something in the gcc man pages).  I wasn’t really in the mood to learn yet another coding IDE, but I did want to post some of the gotcha’s I found in creating the Mac version of the ANE.  These are not step-by-steps, but rather things that tripped me up.

  • When you create a new Xcode project, you must choose a Cocoa Framework project (under Framework & Library).
  • Add the “Adobe AIR.framework” by clicking on the project name, and then going to Add, Existing Frameworks.  The Adobe AIR Framework file is found in your “Flash Builder App Dir -> sdks -> 4.6.0 -> runtimes -> air -> mac”  folder.
  • The Adobe AIR Framework file auto-includes the header file, and the external library that you need to link to.  Don’t go searching for another header file, you don’t need it.
  • Create/Import your .c and .h files and attempt to compile before setting your project settings.  If you don’t try to compile before you set your project files, you won’t see 3/4 of the compiler settings you need to make things right.  This one threw me into a loop for at least a day.
  • All of the examples have you including the <Adobe AIR/Adobe AIR.h> file in your project — I found this works, but all it is doing is including the FlashRuntimeExtensions.h that exists within the Adobe AIR framework.  I decided to resist change and include the Adobe AIR.h file.  It works.
  • There are two places you need to set your compiler and project settings.  One is on the project itself (the top-most node in the Groups & Files tree), and the other is in the Targets -> ProjectName.framework node.  Set all settings in both places (they don’t cascade once it has been compiled once, which you have to do to get the GCC settings to show up right).
  • Make sure your project is compiling 32-bit frameworks, targeted to the 10.5 SDK.   I never found that documented anywhere, and it was sheer luck I tried those combos.
  • Make sure that you turn the optimization level to None (0).  If you don’t do this, things will seem to work, but if you use any of the FREGet or FRENew functions, they will produce seemingly random numbers (I was convinced it was passing me back pointers casted as integers, but who knows).
  • Make the following target and project settings changes that deviate from the defaults :
    • Architetures : 34-bit Universal
    • Base SDK : Mac OS X 10.5
    • Build Active Architecture Only – Checked
    • C/C++ Compiler Version : GCC 4.2
    • Linking Compatibility Version : 1
    • Linking Current Library Version : 1
    • Other Linker Flags : -flat_namespace -weak_Framework “Adobe AIR”
    • Runpath Search Paths :
      • @executable_path/../runtimes/air/mac
      • @executable_path/../Frameworks
      • /Library/Frameworks
    • Packaging Framework Version: A
    • Wrapper Extension : framework
    • Framework Search Paths :
      • /Applications/Adobe\ Flash\ Builder\ 4.6/sdks/4.6.0/runtimes/air/mac
    • Library Search Paths: /Library/Frameworks/Adobe\ AIR.framework/Versions/1.0
    • GCC 4.2, Generate Position-Dependent Code: Unchecked
    • Optimization Level : None [-O0]
    • Other C Flags : -fvisibility=hidden
    • Other C++ Flags:  -fvisibility=hidden
    • Precompile Prefix Header : checked
  • In order to export the proper function names for your initialization and finalization functions, prefix them with __attribute__((visibility(“default”)))

Once you are creating your .framework properly, you should be all set.  If you created truly POSIX or standard C code, there should be very little changes you should need to make your project work.

I do have to make one additional note about a bug people have been finding with Flash Builder 4.6, as it was released on 11/30.  If you are attempting to include an ANE into a project, the project will compile, but if you debug the project, it will complain that it can’t find the method name (not the initilizer or finalizer, but one of your other method names).  If you debug your project from ANT or the command line, it will work fine.  The work-around for this is the following steps :

  • Make sure you have ADL in your search path.  Edit the /etc/paths file and add a new line with /Applications/Adobe Flash Builder 4.6/sdks/4.6.0/bin
  • Create a new folder on your desktop, Document folder, or somewhere where you will find it.  Copy your .ANE into it.
  • Create a new folder within this folder, and name it MyTestANE.ane    (make sure to include the .ane — that is very important)
  • cd into that new .ane directory, and execute the following command :    unzip ..\ArduinoConnector.ane      (assuming you named your .ane ArduinoConnector.ane)
  • Go into the bin-debug folder of your project, and run the following command :    adl -profile extendedDesktop -extdir ~/Documents/Extensions/ Main-app.xml    assuming that the directory you created in the second step was named Extensions and was located in your Documents folder.
  • You can repeat the last step as often as you need with no additional work, assuming you don’t need to update your .ane.   If you need to attach the debugger / profiler , you can follow the steps outlined on this blog (but use about:blank instead of http://localhost)

Well, I think that is about it.  I hope I can help alleviate some of the frustration that I suffered for somebody else in working with all this :)

Connecting your Arduino to AIR using an AIR Native Extension

One of the things that has been bugging the the most when working with Arduinos and AIR is the fact that you needed some sort of proxy application running in order to allow the AIR (or Flash) application to talk to the Arduino.  You see, the Arduino only exposes a Serial port (via USB), and AIR is limited in this regard.  The way people have gotten around this so far is to have AIR create a TCP socket connection to some sort of Proxy application, which in turn opens the Serial port to the Arduino.  It works, and it works very well, however this is a separate application you have to ship, have the user configure, and run.  Heck, even me as the developer found this solution clumsy when demoing these solutions.

I finally decided to fix the problem and write an AIR Native Extension.  The AIR Native Extension (ANE) is a C based .DLL / .framework for the Windows/Mac platforms that allows AIR to essentially open a COM port.  I wrote it in a way that is supposed to emulate the functions of the flash.net.Socket library that is included in the AIR runtime.  I’ve posted the entire project, including the source code and final binaries on Google Code at http://code.google.com/p/as3-arduino-connector/ (well, everything except for my compilation scripts, which are specific to my computers). 

The biggest learning experience in creating this ANE was developing on the MacOS platform.  I’ve never done any programming targeted for that platform before, and working with XCode is just a pain.  I’m used to environments like Visual Studio and Eclipse, but XCode always seemed to fight me every step of the way.  From simple things like hiding all the project properties, to trying to force you into an MDI workflow.  Also, working with the lack of documentation on the AIR Runtime side was kind of depressing…  Don’t worry, a future blog post will try to fill everybody in on how to make an ANE using XCode.

Using the ANE :

  • Include the ANE into your project.  Make sure you are using Flash Builder 4.6 or later. Right-Click on your project in the Package Explorer (Project Explorer), and go to Properties.  Go to the Flex Build Path tab, and then the Native Extensions tab.  Click “Add ANE…” and bring it in.  It does not need to reside within your project source folder.
  • Next, import the com.quetwo.Arduino.ArduinoConnector and the com.quetwo.Arduino.ArduinoConnectorEvent packages.
  • Instantiate a new variable of type ArduinoConnector.
  • Check the isSupported() function to make sure that the ANE is supported on your specific platform, and if it loaded properly.
  • Call the getComPorts() function which will return an array of valid list of COM ports on your OS.  On Windows, this returns ALL COM ports that are valid in Windows, where on the MacOS platform, it will return any USB Serial Devices, which usually would only be your Arduino.
  • Make the connection to the Arduino by calling the connect(comPort, baud) function.  You will need to pass in one of the COM ports from the getComPorts() array, along with the baud rate that your Arduino is operating on.  If you are using Firmata, it is 57600.  Most other Arduino projects use 9600.
  • Next, add an event listener to listen for the “socketData” event.  This will fire when new data is available in the data buffer.  Don’t wait too long to pull data out of the buffer, because it is only 4k :)

Sending Data:

Sending data is just like the Socket class, except you don’t have to flush() the buffer — the data goes out in realtime.  The two most common ways to send data :

  • writeString(“hello world”);
  • writeByte(255);

Getting Data from the Buffer:

As data arrives and is placed into the buffer, the bytesAvailable variable is incremented to reflect how many bytes are available.  Most people will read data from the buffer using one of the two functions :

  • readBytesAsString();
  • readByte();

All of the read functions are FIFO, meaning they will return the oldest data in the buffer before they return the newest (First In, First Out).

On the Google Code site, I am posting more detailed documentation, including a simple patch to as3Glue which will allow as3Glue to work seamlessly with this ANE. 

Enjoy, and please provide feedback on this ANE.  It’s the first one I’ve released to the public and I’d like to know how it works for everybody!

Creating a Windows AIR Native Extension with Eclipse – Part 3

In part 3 of this video series, I talk about how to write the AS3 portion of the AIR Native Extension, and how to package all the components into your final .ANE file. 

You can watch part one here, and part two here.  Part four will talk about how to use the .ANE within Flash Builder.  I will post all the files used in this tutorial in the final blog post.

Creating a Windows AIR Native Extension with Eclipse – Part 2

In part two of this video series, I go through how to actually program your ANE Windows DLL.  This involves doing some C programming.  Please see part 1 here.

The snippets mentioned in this video are available here :  ANE Snippets Download     You can use these to jump-start your development.

Follow

Get every new post delivered to your Inbox.

Join 29 other followers