ACT-CV - Machine Vision for Cognitive Modeling
This package is intended to create an easy to use connection between arbitrary graphical user interfaces and a cognitive modeling tool.
Four types of connection targets have been implemented
The screen images/videos are analyzed with OpenCV. The following demos are currently available:
The package consists of four elements:
This algorithm works quite straightforward. Use this one to check whether ACT-CV works well.
ACT-CV currently returns the content of the following HTML tags:
A- link anchor
DIV- generic divisions
TD- table cells
INPUT- input elements, e.g. buttons
OPTION- entries in a choice group
The returned position is the rectangle used by the complete element, not just the text within. The relative positioning of the text (top, bottom, center, ...) is currently not taken into account.
You must provide
You can find sample files in the
imgs directory of the source code distribution.
The definition file has one line per match target. Each of these lines consists of three entries:
VALUEslot of the
visualbuffer chunk that represents this object when you use the ACT-CV device for ACT-R.
ACT-R is an elaborate architecture for the generation of cognitive models. ACT-CV has been designed for the use with ACT-R.
ACT-CV depends on the CFFI package, which is available for all major Lisps. This way, ACT-CV is not bound to a single flavor of Lisp. After having loaded CFFI, you can load the file "act-cv-dev.lisp" from the actr-device directory of the distribution. There is a sample ACT-R model in "act-cv-test.lisp" in the same directory that demonstrates how ACT-CV works.
ACT-CV is distributed under the terms of the GNU General Public License. See the file LICENSE in the root directory of the source code distribution.
Download the latest version of ACT-CV from http://sourceforge.net/project/showfiles.php?group_id=214599
For building ACT-CV, you must have the following software packages
If you want to use ACT-CV with Lisp, you need the CFFI package for Lisp: http://common-lisp.net/project/cffi/
The Lisp interface of ACT-CV has been generated using SWIG (http://www.swig.org/), but you will only need this if you want to make changes to the interface.
ACT-CV comes with a client/server version that allows to run image processing and the software system that uses the results of the processing (i.e. the cognitive model) on different machines. Client and server communicate via ICE (http://www.zeroc.com/ice.html).
aclocal ; libtoolize ; autoconf ; automake -a -c
./configure [CXXFLAGS="-I /path/to/Ice/include -L /path/to/Ice/lib"]
configure should find both your Java and OpenCV installation automatically. If not, please direct it to them by providing additional directories to c++ (as shown above).
cmake (cross platform make: http://www.cmake.org) for generating MS Visual Studio files. During the cmake call you have to point out where your Java and OpenCV headers and libraries are located.
Parts of this work have been supported within the DFG funded project "Automatische Usability-Evaluierung modellbasierter Interaktionssysteme für Ambient Assisted Living", see also http://www.tu-berlin.de/?id=135088
Parts of this work have been supported within the DFG excellence initiative research cluster "Cognition for Technical Systems - CoTeSys", see also http://www.cotesys.org
Halbrügge, M. (2015). Fast-Time User Simulation for Dynamic HTML-based Interfaces. In Taatgen, N. A., van Vugt, M. K., Borst, J. P. & Mehlhorn, K. (Eds.), Proceedings of the 13th International Conference on Cognitive Modeling, pages 51-52, Groningen: University of Groningen
Halbrügge, M. (2013). ACT-CV: Bridging the Gap between Cognitive Models and the Outer World. In Brandenburg, E., Doria, L., Gross, A., Güntzler, T., and Smieszek, H. (Eds.), Grundlagen und Anwendungen der Mensch-Technik-Interaktion. 10. Berliner Werkstatt Mensch-Maschine-Systeme, pages 205-210, Berlin: Universitätsverlag der TU Berlin
Halbrügge, M., Deml, B., Färber, B. A. & Bardins, S. (2007). ACT-CV - Die Erweiterung von ACT-R um Bildverarbeitungsalgorithmen erlaubt die schnelle Erzeugung mächtiger Benutzermodelle. In In: Grandt. M. & Bauch, A. (Eds.), Simulationsgestützte Systemgestaltung - DGLR-Bericht 2007-04, pages 313-331, Bonn: DGLR e.V.