FlexUnit 4 assert failures have...

With FlexUnit 4 in Flash Builder 4 Plugin Beta, I created 1 suite with 1 class with 1 default failing test method. When I ran it, the assert failure had a call stack 60 methods high. Is this a bug?

The stack trace has several ''async'' methods, e.g. applyExplosivelyAsync(). This seems strange because?I'm not using async at all. Is this intended behavior?

Robert


Test method Not yet implemented

at flexunit.framework::Assert$/

fail()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\fl uint\FlexUnit1Lib\src\flexunit\framework\Assert.as:370]
?at WidgetTest/testWidget()[C:\Users\robert.penner\Adobe Flash Builder
Plug-in Beta 2\TestProject\src\WidgetTest.as:20]
?at Function/http://adobe.com/AS3/2006/builtin::apply()
?at flex.lang.reflect::Method/apply()[C:\Users\mlabriola\Documents\workspaces\net\d igitalprimates\opensource\flexunit\FlexUnit4\src\flex\lang\reflect\Method.as:124 ]
?at ReflectiveCallable/run()[C:\Users\mlabriola\Documents\workspaces\net\digitalpri mates\opensource\flexunit\FlexUnit4\src\org\flexunit\runners\model\FrameworkMeth od.as:214]
?at org.flexunit.runners.model::FrameworkMethod/applyExplosivelyAsync()[C:\Users\ml abriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\s rc\org\flexunit\runners\model\FrameworkMethod.as:121]
?at org.flexunit.runners.model::FrameworkMethod/invokeExplosivelyAsync()[C:\Users\m labriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\ src\org\flexunit\runners\model\FrameworkMethod.as:132]
?at org.flexunit.internals.runners.statements::InvokeMethod/evaluate()[C:\Users\mla briola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\sr c\org\flexunit\internals\runners\statements\InvokeMethod.as:52]
?at org.flexunit.internals.runners.statements::StackAndFrameManagement/evaluate()[C :\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\F lexUnit4\src\org\flexunit\internals\runners\statements\StackAndFrameManagement.a s:94]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.token::AsyncTestToken/sendResult()[C:\Users\mlabriola\Documents\wo rkspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\toke n\AsyncTestToken.as:70]
?at org.flexunit.internals.runners.statements::AsyncStatementBase/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\AsyncStatementBase.as:44]
?at org.flexunit.internals.runners.statements::StatementSequencer/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:103]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:86]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.runners::BlockFlexUnit4ClassRunner/runChild()[C:\Users\mlabriola\D ocuments\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\fl exunit\runners\BlockFlexUnit4ClassRunner.as:104]
?at org.flexunit.internals.runners::ChildRunnerSequencer/executeStep()[C:\Users\mla briola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\sr c\org\flexunit\internals\runners\ChildRunnerSequencer.as:49]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.token::AsyncTestToken/sendResult()[C:\Users\mlabriola\Documents\wo rkspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\toke n\AsyncTestToken.as:70]
?at org.flexunit.internals.runners.statements::AsyncStatementBase/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\AsyncStatementBase.as:44]
?at org.flexunit.internals.runners.statements::StatementSequencer/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:103]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:86]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.runners::ParentRunner/run()[C:\Users\mlabriola\Documents\workspace s\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\runners\Par entRunner.as:315]
?at org.flexunit.runners::Suite/runChild()[C:\Users\mlabriola\Documents\workspaces\ net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\runners\Suite .as:59]
?at org.flexunit.internals.runners::ChildRunnerSequencer/executeStep()[C:\Users\mla briola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\sr c\org\flexunit\internals\runners\ChildRunnerSequencer.as:49]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.token::AsyncTestToken/sendResult()[C:\Users\mlabriola\Documents\wo rkspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\toke n\AsyncTestToken.as:70]
?at org.flexunit.internals.runners.statements::AsyncStatementBase/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\AsyncStatementBase.as:44]
?at org.flexunit.internals.runners.statements::StatementSequencer/sendComplete()[C: \Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fl exUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:103]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:86]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.internals.runners.statements::StatementSequencer/executeStep()[C:\ Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\Fle xUnit4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:62]
?at org.flexunit.internals.runners.statements::StatementSequencer/handleChildExecut eComplete()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensour ce\flexunit\FlexUnit4\src\org\flexunit\internals\runners\statements\StatementSeq uencer.as:82]
?at org.flexunit.internals.runners.statements::StatementSequencer/evaluate()[C:\Use rs\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexunit\FlexUn it4\src\org\flexunit\internals\runners\statements\StatementSequencer.as:68]
?at org.flexunit.runners::ParentRunner/run()[C:\Users\mlabriola\Documents\workspace s\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\runners\Par entRunner.as:315]
?at org.flexunit.runner::FlexUnitCore/beginRunnerExecution()[C:\Users\mlabriola\Doc uments\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flex unit\runner\FlexUnitCore.as:177]
?at org.flexunit.token::AsyncListenersToken/sendReady()[C:\Users\mlabriola\Document s\workspaces\net\digitalprimates\opensource\flexunit\FlexUnit4\src\org\flexunit\ token\AsyncListenersToken.as:59]
?at org.flexunit.runner.notification.async::AsyncListenerWatcher/sendReadyNotificat ion()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\fle xunit\FlexUnit4\src\org\flexunit\runner\notification\async\AsyncListenerWatcher. as:87]
?at org.flexunit.runner.notification.async::AsyncListenerWatcher/handleListenerRead y()[C:\Users\mlabriola\Documents\workspaces\net\digitalprimates\opensource\flexu nit\FlexUnit4\src\org\flexunit\runner\notification\async\AsyncListenerWatcher.as :97]
?at flash.events::EventDispatcher/dispatchEventFunction()
?at flash.events::EventDispatcher/dispatchEvent()
?at flexunit.flexui::FlexUnit4TestRunner/handleConnect()[C:\work\flex\ide_builder\c om.adobe.flexbuilder.flexunit\flexunitframework\src\flexunit\flexui\FlexUnit4Tes tRunner.as:296]
?at flash.events::EventDispatcher/dispatchEventFunction()
?at flash.events::EventDispatcher/dispatchEvent()
?at flash.net::XMLSocket/reflectEvent()FlexUnit 4 assert failures have...

Not a bug and fully expected behavior.

FlexUnit 4 is really just a collection of runners that can handle just about anything that is thrown at it as well as handle any pluggable runners that others write. The level of extensibility comes with some complexity. So, by the time that single method of yours is executed, there are actually probably a hundreds methods in the framework for getting everything setup, inspected and ready to go.

The stacktrace always looks up the stack from your failure to the last point where a particular chunk of code execution began, so the 60 is basically just that number of methods. There isn't a whole lot that can be done about it save for eventually provinding filtering to eliminate showing some of our stuff and making the track more legible.

Regarding the async thing, your method isn't async but FlexUnit4 is. Inside of the Flash Player we are operating on frames. We don't have the luxury of threads and the like that Java has, so we have to handle the possibility that something might be async on a much deeper level. We also do something called green threading where we actually watch how much time we take in execution. Once we have taken 90% ish of the time available in a given frame, we wait until the next one before we begin executing again. This allows us to run long and compelx test suites without getting a timeout from the flash player (which happens if any single frame takes longer than 15 seconds to execute). So, even though your method isn't doing anything async, all of these other pieces have to account for the chance that something will be. Hence the reason that you see async pieces in the stack trace.

Hope that helps,

Mike

FlexUnit 4 assert failures have...

Thanks for the explanation, Mike. Pluggable runners sound interesting--what's being worked on in that area (you or others)?

Robert

I am curious, though--what does ''explosively async'' mean?

Another question: If I wrote my own minimalist test runner, where would I plug that into FlexUnit? What would the key interfaces be? How much of that stack trace could I potentially eliminate and how much would necessarily remain if I plug into FlexUnit?

The FU4 and FU1 support as well as theories and parameterized tests are all actually pluggable runners already. In addition, the community has written pluggable runners that support FUnit, Structured Log Testing and a few other testing frameworks which will be made available soon (projects not under my control so I don't think I should comment on them just yet). It effectively means that FU4 is responsible for identifying a test but then hands off execution to a separate class.

To be a pluggable runner, you simply need to implement 1 interface (org.flexunit.runner.IRunner) and be able to provide the descriptions and hand-shaking required. You specify that you wish to use a plugable runner via the RunWith metadata

You could absolutely write your own to eliminate some of the stacktrace overhead (but there would still be some while FU4 introspects the classes and figures out that you intend to run in a separate runner).

Mike

invokeExplosivelyAsync basically means to invoke the method, not automatically assumign that the end of the method call is the end of the test, and to anticipate that it just might explode

Mike