Final Post for CREU '16-'17 group

"How lucky I am to have something

that makes saying goodbye so hard."

It is with a great deal of bittersweet emotion that we say goodbye. Last Thursday, May 4, marked the last meeting of our research group. We discussed the work that still has to be completed, like analyzing the data from the user surveys. Over the course of the summer, Nico will analyze the data we collect and use that data to write another research paper.

This past academic year has challenged us in ways we never imagined and allowed us to grow as students, researchers, software developers, and collaborators. Overall, our group is very pleased with our accomplishments and we are extremely thankful for the opportunity to participate in this amazing program.

So long, and thanks for all the fish.

NMT CREU '16-'17 group

(4/28) A brief analysis of work completed

Over the course of the program, Hugo, Eugene and I have learned much more than I think any of us expected to about the research process, mobile development, databases, and creating a RESTful API server. This post will serve to summarize the work that we've done on the server, and where our research will go from here.

We have written 3845 lines of code for the server. About 92% of this code is Java, leaving about 8% of the code as HTML/CSS. We have made 201 commits to our master branch for the server, which represents about 1 commit per day. The graph of these commits shows a clear spike as we neared the start of the experiment, and another spike as we ran into bugs that prevented us from testing in early April. Despite these spikes, the graph also shows that we have been consistently working on the server since early October.

The CREU program ends on 5/5, which is the same day that Eugene and I will oversee the last session our app is used. Once the final session is complete, we will issue an exit survey, which asks students to rank the methods of authentication and asks questions about the usability of the application. After this, we will no longer have funding, but Dr. Zheng and I have arranged for me to take Directed Research course over the Summer to analyze the data we have collected from the experiment, write a second research paper, and to write any documentation necessary for a future CREU team to continue to use our application for their research. This CREU team could add notifications to our application to study the effect of personalized feedback on student attendance rates, or could move forward with authentication using facial recognition or location determination via Wi-Fi networks.

In all, I'm extremely grateful for the opportunity to have worked on this project. This is definitely the longest I have ever worked on a single software project, the largest codebase I have ever worked with, and the most demanding in terms of my time. I have learned much more than I thought I would, gotten a feel for the research process, and worked with some really amazing people.

--Nico

(4/23) Another hiccup

This is the fifth in a series of backdated blog posts.

Our application now works with all methods, though we did suffer another setback last week. We host the server component of our application on a virtual machine managed by NMT's Computer Science department, which had its storage machines go down Monday of this week. Because of this, many machines' hard drives went into read-only mode, including the machine hosting our server. Even though our machine was restarted, our Tomcat instance was not, and so no students were able to sign in Monday. We compensated for the delay by holding two sessions, so this didn't affect our schedule.

Our application is also in the review process for the Apple App Store, which means that, proctoring the last few sessions and performing data analysis aside, we've basically reached the end of our project. I'll make a post next week detailing the work done on the server side, and our continuing research after the end of this year's CREU program.

--Nico

(4/16) Preparing for the SRS

This is the fourth in a series of backdated blog posts.

Work on the server has largely been completed, so Eugene and I spent this week preparing our poster for the SRS using the LaTeX template that Hugo created for us. We've completed the test sessions for both PIN and Password, and will probably be conducting two more tests for PIN and Password to give Hugo time to resolve UI bugs caused by adding the QR method.

The SRS takes place on Thursday, April 20, and we will be presenting our poster to passers-by and judges. They will submit surveys indicating their understanding of our research and our clarity in presenting.

--Nico

(4/9) Hiccups in the experiment

This is the third in a series of backdated blog entries.

As Eugene has mentioned already, the experiment schedule is being pushed back a week.

Before Monday, the intended first day of the experiment, Hugo, Eugene and I worked hard to finalize the application. We generated user accounts, finalized the communication protocol between the client and server, and simplified the instructor's web application. However, I had relied on behavior that was previously in our communication protocol to determine the course that the user was in, and because I tested the login and attendance submission process using a REST client, was unaware that the application was not sending the course number that the students were in. As such, on Monday, no students were able to log in. The fix was relatively simple-- we could just look up the student's course number in the database, as intended.

On Wednesday, we encountered another bug. The client expected that the position of a student's course in the list sent  would also be the course's ID number, but this was true only for accounts enrolled in our test session. This caused everything to look like it was working as we were testing our changes, but when students enrolled in the actual classes logged in, they could not submit attendance to the correct class because it was greyed out.

We fixed these issues before Friday and conducted a successful test in one class's lab session, which was a welcome relief.

We also enabled TLS for both the client and server before Wednesday, which was a difficult process for Hugo and I. The solution on my end was to stop attempting to enable TLS using Tomcat, which is needlessly complex, and to instead use an NGINX reverse-proxy.

In all, both setbacks were frustrating and time-consuming to resolve, but didn't affect our experiment too much.

--Nico

(4/2) Beginning the experiment

This is the second in a series of backdated blog entries.

We began the experiment Monday, beginning with taking attendance traditionally. This is being done both to give students a point of reference for when we start to use our application, and to collect data on the time our application saves over traditional methods, if any.

Next week, Eugene and I will introduce the PIN and Password methods of authentication. The following week, we will introduce the QR and Pattern methods. The week following this, we will require participants to use PIN and Password unassisted, and the next week, require them to use QR and Pattern unassisted. The experiment is being structured this way so that we collect data from the unassisted sessions in order to eliminate the effect that unfamiliarity with the application may have on submitting attendance.

--Nico

(3/26) Return from conference and Spring Break

I have some catching up to do-- this is the first in a series of backdated posts detailing what I've done in the month since we returned from our conference.

Eugene's conference debriefing post contains most of what we did during SITE 2017, but he did mention that we had to take some time the day of our presentation to work more on our application.

As we were preparing to present, we needed to conduct a final test of our application to confirm everything was working correctly. Unfortunately, we were suddenly unable to record our attendance in the session even though everything had worked perfectly just two days prior. It took about 30 minutes to discover that we had been using the phone's local time to determine whether a session was open, or whether students were on time, late or absent-- this meant that with the shift to Austin's time zone, no class sessions were seen as available! This also could have allowed students who were late or absent to manipulate their phone's clock to appear as if they were on time, so we were glad to discover this bug, even at such an inconvenient time.

Once again, we're very grateful to CREU for funding our travel and to Dr. Kuo for accompanying us to SITE 2017.

--Nico

SRS Update

I adapted a template poster written in the LaTeX typesetting language so my teammates and I could present at the Student Research Symposium at our university.
I created an outline of our work and discussed the poster's contents with Dr. Zheng and Dr. Ku, and I adjusted the poster's typographic features in order to squeeze in as much content into our limited 48 by 36 inch poster as possible.
A notable feature is our use of microtype in order to improve the appearance of the text using several techniques. These are masterfully described by Khirevich here: khirevich.com/latex/microtype/

Our presentation was Thursday, April 20th. Like our oral presentation at the SITE conference in Austin, our poster presentation inspired several useful comments from the educators in our audience. A highly desired feature was the ability to use our app for quizzes.

- Hugo

What I've Learned About Qt

I'd like to share some thoughts regarding the Qt framework, which we used to develop our cross-platform application. It sports an amazing feature set: any features that have become standard in a grpahical user interface are easy to implement. For example, a slide-out drawer menu as seen in many mobile applications is an easy to add feature in Qt. 

The Qt framework is written in C++, a flexible, efficient, and cross-platform
language. Thanks to the Android NDK C++ compiler and Apple's Clang C++ compiler, our Android and iOS apps benefit from easily integrated external libraries that provide advanced features. This includes the QR decoding library quirc, which I modified in order to use it to decode images captured using Qt's camera interface. excluded from the Qt framework some examples:  quirc for Qr decoding and mbedtls for cryptographically secure communications

The biggest drawback to the Qt framework is its large size. Our Android users must download an 11MB APK file when they first install the application from the Google Play store. This size delays updates and slows down our development. It also increases the complexity of our final product, a detriment to any security evaluations or debugging sessions. Although I attempted to enable our compilers' dead-code elimination features, which would remove any unused components and therefore produce a much smaller installation file, this feature is impossible to perform automatically with today's technology. This is due to our use of the programming languages QML and Javascript, the compiler cannot easily prove that these dynamic languages will nevery require a given Qt feature. Therefore the installation file must include the entirety of the Qt framework.

C++ and QML are difficult languages with unfriendly syntax (compared to Python) and fragile type systems that cannot prevent as many errors as the more advanced and consistent type systems of other languages, such as Haskell, Coq, Ocaml, Rust, and Java. Unfortunately, our team had to deal with several runtime errors that could've been prevented by a more advanced language.

- Hugo

Development Update: Features We Didn't Deploy and Things We Didn't Do

Although these features received some development time, we were unable to release them in time for our user study.

1. Countdown timer that displayed how long users had before their class began and how long they had until they were marked tardy or absent.
2. A detailed attendance log viewer so users could see a thorough list of their attendance reports during our user study.
3. A more efficient data representation using Google's protobuf data format. This would enable our client and server to communicate more rapidly than by transmitting data using the JSON format.
4. A faster database, such as MariaDB, would've let our server save and load data at a much faster rate.
5. A constant integration system that reported any compilation failures once we uploaded either client-side or server-side code to our code versioning system.
6. Unit tests for the UI so we could guarantee that users had access to given functionality.

The first 3 features required more thorough debugging, but we did not have time to finish them. Features 3 and 4 were not necessary as our server responded fast enough, with a mean latency of 50ms and no waits longer than about half a second. If we had more users communicating with our server using either the Android app or the iOS app, we would have to implement several optimizations such as features 3 and 4.

The last two features were sorely missed as we occasionally deployed updates that lacked key features. Any professional software development team would have implemented these in order to ensure that users would not be exposed to undue feature regressions.

- Hugo

Visual Design Update (04/26)

I spent about a week designing the application's icon. It remains visible on various backgrounds and avoids use of color combinations that would prevent color-blind users from identifying our application. I also wrote a Python script that converts the vector-graphic into a raster graphic of several dimensions. This was necessary because Android apps must contain the application icon in several dimensions in order to display high resolution icons to the diverse set of devices. Android icon files must be 24 by 24 pixels to 512 by 512 pixels. iOS has similar size requirements.



The application's user interface benefitted from Google's Material Design principles. I modified early prototypes in order to keep UI elements close to each other, and to use depth in the 2D user interface in order to follow the Material Design's paper metaphor. For example, buttons appear higher than text input elements, and the menu drawer appears to cast a shadow from an even greater height. In order to improve accessibility, I implemented a font size changer. This was required because Qt ignores the system-wide font size setting. However, we neglected to use Qt's accesbilities features which would've made our program more readable to screen reading devices.

- Hugo

Our app on an Android app launcher.

The login screen

The slide-out menu.

The help screen.

Need to change your password?

A course list (with a single course)

Result of reporting attendance.

Taking a picture of a QR code to report attendance.

QR code detected!

Exiting the application

Development Update (04/26)

Our server is responsible for receiving and storing courses, users, attendance logs, and attendance sessions.

My code generator ensures that the client and server are operating on data using a consistent naming scheme: a "User" object on the client is the same as a "User" object on the server, and they are both written to and read from our SQLite database efficiently. I created the data objects used in the client and server and I also setup the  SQLite database code in the server. Nico and I designed these data objects through a series of discussions about what kind of data we would need to communicate.

I also improved server security by enabling certain software configurations that tend to improve the confidentiality and integrity of the information we must store. This is in conjunction with improved monitoring by using a log viewing tool. This lets us view a live stream of who is communicating with our server.

- Hugo

What Students See

Students participating in the user study for our attendance tracking system can download the application by visiting an opt-in URL that takes them to the Google Play Store. They download a 10 to 12 MB APK file and wait for the installation process to finish.


- user logs in, picks a course, sends the appropriate authentication method
- in case of connection error: try to show a diagnosable error message
- sends salted hash of password to the server
- receives course list (typically one course)
- user picks a course and sends the appropriate
- pattern, password, pin are all QML
- QR: quirc computer vision library, QML camera
- tried to process full image, instead use C++ to access and process preview image
- QR croppableimage in QML
- compile and upload to play store (we don't have CI)

- Hugo

Security Update (04/26)

Our first time users must read the privacy policy before they can use the app. They are required to answer some short questions in order to demonstrate knowledge of the data that we require for our user study and of the data that we exclude from communications with our server.

1. we only collect anonymous research IDs, only instructors can correlate attendance logs with actual student names
2. we do not continously collect information, students send us information only when they click one of two specific buttons
3. we never transmit images through the network; QR codes within the image are decoded and the payload of the QR code is sent instead of the image itself

We use transport layer security (TLS, footnote: this used to be done with a now-obsolete technology known as SSL) in order to preserve data confidentiality (prevent snooping) as our application communicates with our server. This means that anyone "listening in" on the user's network traffic would only see a stream of random data, which could only be decrypted by our server.

The Qt networking library (We use networking libraries in order to send and receive data from our server.) helps us achieve encrypted communication on the iOS and MacOS platforms and the mbedtls networking library does the same on Linux and Android. Ideally, we would've used the Qt networking library for all of our platforms, but incompatibilities between Qt and Android prevented us from doing so.

My debugging sessions led me to discover that Android would load the of OpenSSL networking library present on the Android device instead of the OpenSSL networking library that we packaged for the Android application. This means that Android would prefer to use the software located on the user's phone (these are wildly different networking libraries), as opposed to loading a version of software that would be compatible with the Qt networking library.

So mbedtls, a cross-platform TLS networking library, became our Android and Linux networking library. Due to difficulties with the iOS build system, we continued to use the Qt networking library on Apple devices.

- Hugo

Development Update (04/26)

We have deployed and continue to maintain a complex program with about 40
users. Although data collection for our user study is still ongoing, this project taught us many valuable lessons regarding software engineering, user interfaces, and most importantly: communication.

The following blog posts are a report of the code I have written for this research project. We continue using the Qt framework to write a cross-platform application in the C++, QML, and Javascript programming languages. Our attendance taking application consists of about 1,800 lines of C++ code that is used for efficient networking, cryptography, QR code decoding, and other data processing; and about 2,700 lines of QML and Javascript code for describing the graphical layout and interactive elements of our app.

We share our code by uploading to our school's gitlab code versioning server and we discuss issues by using the Riot messenger and our school's issue tracking service.

- Hugo

Development Update: 03/16

We have been performing stress tests on our server. Eugene and I designed and implemented a testing script that starts 20 parallel threads that send various requests to the server as fast as possible. The tester sends correct and incorrect information, such as login requests and dummy attendance reports. The testing data is randomly generated --- for example, the script sends extremely large payloads (1MB) or valid and invalid usernames.
We have run the test concurrently on at most 4 machines (80 total connections). Our server has a response time of 100ms with a standard deviation of 50ms and max/min response times of 1000ms and 7ms. We'll use our newly developed stress testing script to evaluate several upcoming server-side changes and optimizations.

- Hugo

Update 4/23/17

Work accomplished:
Our group's poster presentation at the Student Research Symposium was a success!

Our group with our poster

Explaining our project to a group of our peers

Presenting to a NMT professor

Our project received a lot of positive feedback. One thing instructors would like to see in the application is a quiz authentication method -- where the instructor can ask a question and receive a response as an evidence type. Another instructor also suggested some form of integration with the iClicker system.

Goal:

Begin the unguided attendance sessions.

Outcome:

From these unguided sessions, we will ultimately gain an understanding of how the users feel about the system in a real-world situation.

-- Eugene

Update 4/16/17

Work accomplished:
Nico and I have demonstrated the PIN and password authentication methods. We allowed the study participants to ask any questions they might've had regarding the two methods. Our group has also completed the poster we will present at the SRS.

Goal:
In this next week we will demonstrate the two remaining methods. We will also be at the SRS on 4/20 from 11:20 - 12:30.

Outcome:
The participants will be familiar with the pattern and QR authentication methods. We will strengthen our presenting skills from our presentation at the SRS.

-- Eugene

Update 4/9/17

Work accomplished:
I have created a password generation script that will create random passwords for each study participant. The participants will be able to change their password once they have logged into the application. I have also wrote some code that inserted the user/password combinations into our database. We attempted to use the application on Monday, however there were some underlying issues that rendered the app unusable. After we thought all issues had been fixed, we attempted to test the application again on Wednesday with no luck. The server-side and client-side issues have been resolved and we had a successful test session on Friday.

Side note -- Our paper is now available here on the learntechlib website!

Goal:
Nico and I will demonstrate the PIN and password methods to the classes.

Outcome:
The participants will have a better idea of how to use the system and they will have the opportunity to ask any questions they might have.

-- Eugene

Update 4/2/17

Work accomplished:
We started the experiment on Monday and we began collecting data. For the first week of the experiment the instructors will use traditional attendance-taking methods.  One of the classes used a "quiz" method -- the instructor hands out a piece of paper with questions on it at the beginning of class, and collects them at the end of class. The other class used an attendance sheet method, where a piece of paper is passed around and students write their name. The data we collected this past week looks promising.

Goal:
This week we will begin using our system. I will also begin working on the poster that we will present at New Mexico Tech's Student Research Symposium (SRS).

Outcome:
I have learned what it is like to be a researcher in a classroom environment.

-- Eugene

Update 3/26/17 (Conference debriefing)

Even though we had to take time from developing the system, the trip was a success and we learned so much from attending the conference! The trip allowed us to take a much-needed break and visit the city of Austin, Texas. We began our trip by leaving Socorro on Monday (3/6/17) morning. We arrived at our hotel around 10pm.

On Tuesday we began our day by visiting the County Line restaurant, where we ate our food next to the creek. We then headed to the Sheraton hotel so we could validate our registration and get our conference badges. We spent most of Tuesday attending presentations and learning about the interesting research that was being done by others. After attending presentations we decided to explore downtown Austin, where we stopped by the famous Voodoo Doughnut donut shop. Finally, we returned to the hotel so that we could complete homework that was due that night. 

Our haul from the donut shop

The view while we enjoyed our lunch

With our presentation being Wednesday afternoon, the day up until that point was a blur of preparation and practice. We delivered our presentation successfully and everything went according to plan. There were some questions and valuable feedback from the audience that will be used to improve our system. Sharing a time slot, we had the pleasure of meeting Dr. Jane Hunter and learning about her High Possibility Classrooms. We left Austin on Thursday morning and concluded our trip by celebrating with burgers from the Owl Cafe.

Nico (left), Hugo (right), and myself preparing for our presentation

Our research group delivering our presentation

NMT CREU '16-'17 group at SITE 2017

I'd like to thank the Collaborative Research Experiences for Undergraduates (CREU) program for funding our travel and allowing me to explore this amazing opportunity! I'd also like to thank Dr. Kuo for providing some of the images used in this post.

--Eugene

Update 3/19/17

Work accomplished:
We have returned from our conference trip. There have been some modification to the application user interface (screenshots to come). I have began writing a script that will help us stress test our server. This load testing is important so that we know how the system will behave in a real-world scenario.

Goal:
Over the next week I will finish the script and begin testing our server. I will also write a debriefing for our conference trip.

Outcome:
I have learned a lot from this conference trip. Since this was the first research conference I have ever attended, I learned what it is like to participate in a professional conference and interact with other researchers.

-- Eugene

Update 3/12/17

Spring break

Update 3/5/17

We are in the final stages of preparation for the SITE 2017 conference in Austin, Texas! Come say hello and join us for our presentation!

-- Nico, Hugo, and Eugene

Update for Nico 02/27/17-- Putting it all together

This week Hugo and I have made a lot of significant updates to our group's software. These updates were made to prepare to demo the application for our advisors, and to get the application ready to record a use video which we will present at SITE 2017 on March 09.

Between my last post and meeting with our advisors last Thursday, I rewrote our authentication generators to generate random output, made a webapp to generate attendance sessions with associated random authentication key, made verifying the authentication more generic by making it a simple string-comparison between the sent authorization and the session's auth key, and sent some simple JSON to the application. We presented this to our advisors, who asked that we prepare for a demo this Thursday.

Hugo and I spent several days outlining the JSON-based protocol by which the client and app communicate. Hugo also set up a SQLite database to test with before we move on to a MySQL database in the near future. We worked on the client and server over the weekend, and the server now verifies all four types of attendance methods, checks time stamps, logs in, modifies passwords, generates attendance sessions, displays dynamic course data, has test users for our advisors, uses a database table for all relevant data, no longer uses HTML Basic Authentication, and communicates all of this with the client fully using JSON. The server will also display authentication keys for each attendance session, and possibly also attendance logs for each student, before the end of today.

This means that we should be ready for a full demonstration-- the only issue is that some unknown configuration issue with the Tomcat server results in a crash each time a new .WAR file is uploaded, meaning we will simply demo the app on a local server. Less critically, the non-local server is unable to display generated pictures, meaning QR and Pattern authentication are temporarily unusable.

Besides fixing these two issues, I believe that the only tasks that must be completed between now and the user study are now:

• Fix Pattern auth in the client
• Changes necessary after moving to MySQL
• Modifying our SQL create script to auto-generate student accounts
• Writing a script to correlate research IDs with student IDs, for anonymity
• Discovering whether any students are in multiple course
• Generating a course/user junction table using these data and serving user-specific data to the client
• Sending previous attendance data to the client and displaying these data
• Checking Device IDs

Getting close! See you next week.
--Nico Ponder

Update 2/26/17

Work accomplished:
I have completed the NIH course, and tested the mobile application on iOS.

Goal:
This next week will be an important week since we are preparing to share the details of our work at the SITE conference. In addition to preparing for the conference, I will also begin working on material for the 2017 NMT student research symposium.

Outcome:
From the NIH course I have learned how to conduct human research in an ethical manner, and I understand my obligations to protect the rights and welfare of the subjects involved in human research.

-- Eugene

Update: Surveys and NIH course completion

As Hugo has already mentioned, last Thursday he and I decided on new URLs and their associated GET and POST methods and refined our object model to communicate course and attendance information between the client and server. This is to allow us to conduct a demonstration of an attendance session this Thurday, 2/23, to prepare us for our demonstration at the SITE conference and our user study. I have also completed the NIH Protecting Human Research Participants web course to make our IRB approval process more straightforward and to learn how to ethically conduct our research. I have also revised the entry and exit surveys we will be administering during our research to examine specific vectors more closely.

--Nico

Update: Network Interactions

Last Thursday, Nico and I refined the attendance tracking network protocol. We have taken the object model into account to create the set of resources that the server and client expect to send and receive.

1. GET /login: when the client first connects, the server will send a course list or an HTTP status code indicating some sort of error.
2. POST /create: the client will request a new account by sending an anonymous research ID (which every student in the study will receive) and a password. The server will respond with an HTTP status code to indicate success or failure.
3. GET /attendance_log: the client will send a course ID when the user selects a course. The server will send list of attendance logs or an HTTP status code indicating failure.
4. GET /take_attendance when attendance verification begins, the client will request a list of valid attendance methods. The server will send list of valid attendance methods or an HTTP status code indicating failure.
5. POST /take_attendance: when the user fills out attendance verification, the client will send a JSON object containing verification information. The server will respond using an HTTP status code.

The connection will be performed over HTTPS using HTTP BA for authentication and TSL for confidentiality. Data will be transmitted in the JSON format. I have created prototypes for all the types of networking and data serialization/deserialization that the client and server must perform.  I'll implement our protocol on the client side and I'll help implement part of the networking code on the server.

-- Hugo

Update: SRS

We will present our research at our university's Student Research Symposium (SRS). I have signed us up for a 1 hour long poster presentation. We must prepare a concise description of our attendance tracking system, its design, and our research results by April 20th.

http://infohost.nmt.edu/~srs/

-- Hugo

Update: Approving Our Research

I am submitting an IRB application as the principal investigator of our research project. The IRB is in charge of monitoring behavioral research involving humans; this includes software evaluation. We are avoiding collection of personal information by assigning students anonymous research IDs and we will not restrict our research to adult NMT students. Only the instructors will know the true identities of our participants. We have filled out the IRB Application and Form A and await approval before our tentative start date of March 20th

-- Hugo

Update: Using the System

This blog post summarizes the usage of our attendance tracking system. We have been testing it in order to make it usable for the study, and to keep information safe from leaks.

- it must be usable
- it must keep information safe from any unintended accesses (accidental or penetration, some subversions)

This should make students more willing to participate. The basic procedure for students is:

- download from Play Store (we'll support both x86 and ARM devices)
- or download to their laptop (which can run Linux, Windows, or OSX)
- on initial startup, students associate a password with their anonymous research ID
- students will use this password-ID pair to login on every startup
- students may click the bottom-most button until they reach the attendance verification screen
- students may review their list of classes and corresponding attendance records
- they finish by gathering and sending attendance evidence

The procedure for instructors is:

- download QR or Pattern image
- or copy pin or password
- display this code in a prominent location
- wait for students to register their attendance
- download attendance records as a CSV file from the server

We'll refine this process before we begin our experiment.

-- Hugo

Update: QR Decoding and External Libraries

QR codes are an important feature to the project. I have implemented image loading and QR decoding. Part of the implementation involved familiarizing myself with Qt's build process in order to make use of 3rd party libraries. Qt's image selector is specialized on iOS so that users can select a picture from the native gallery. I used the Quicknative library to provide similar functionality on Android. It uses Qt's Java interface to create Java objects and make calls to the Android API in order to provide users access to a native Android gallery for their image selection. I also used the quirc QR decoding library, which searches a given image for QR patterns using a standard floodfill-based algorithm, then it uses the corners of the QR code to straighten the image with a perspective transform. I had to fork this library in order to add C++11 support and a qmake build file `quirc.pri` so that any Qt project can make use of quirc.  After importing these third party libraries, I parsed the `file:///` path, loaded it as QImage, performed some image processing, converted the image to a character array in order to  run quirc's detection and decoding algorithm. Quirc can process pictures which unambiguous represent a QR code and are  distorted only by a perspective transformation (for example, if the picture was taken from the corner of a room). In case of decoding failure, we are able to send the image to the server.  The next step is to perform QR decoding a separate thread, display a loading bar, and let users crop images before processing.


-- Hugo

Update (1/15): Client UI --- Usability Improvements

We need a usable app that works on many different screens. Now that we have designed the system, I can start refining the UI. The first and easiest way I made the cross-platform UI is by using Qt's Quick Controls. These are prebuilt UI elements that can resize to satisfy simple constraints:  for example, a ListView can be used to show a list of similar elements, and a GridLayout can arrange arbitrary Qt elements in a layout with a certain number of rows and columns.  Another part of the solution involved implementing dynamic margins and element sizes. The application has very small visual margins on smaller screens, but it will not spread out the cell-phone centric UI if it is run on a larger screen. I also implemented a useful features for small screens, the UI will have a minimum screen size after which it will display scrollbars in order to keep all content available. The Qt documentation was useful, I have also been testing and modifying the UI in order to make it easier for our users to access all of the app's features.  

Future goals include using a wider range of testing devices and insuring that all new UI elements work on many different devices.

-- Hugo

Update (1/8): Object Model Updates, SQL+JSON+Java Object+QObject, Storage

An important part of any software project involves translating information from a real-world domain to digital information in the artificial system. We have asked ourselves important questions, including: what do we need to represent? how much information do we need to represent students, attendance logs, and classes? In the Onion architecture, this information belongs in the object model I worked on developing the object model with Nico. Here is a draft of our work (objects are indicated using `*`, each object has a list of property names and types):

* course
- course id (short String like 'CSE 101', unique)
- course name (long String)
- instructor name (long String)
- location (long String)
- readable time (long String)

* users
- research id (long String, unique)
- password (long String)

* student-course junction table

* attendance log
- log id (int, unique)
- received time (timestamp)
- sent time (timestamp)
- auth key (long String)
- auth type (short String)
- auth status (ok, late, absent)
- device id (long String)

foreign keys
- attendance id (int, from the attendance session table)
- research ID (long String, from the users table)

* attendance session
- attendance id (int, unique)
- authtype (short String)
- authkey (long String)
- course id (short String)
- start time (timestamp)
- end time (timestamp)

Now we need to represent the model in code.  The data on the server must be stored in a SQL database, converted to Java Object using `sormula`, and converted to JSON using Google's `gson` library so that it may be sent from server to client Once this data arrives at the client, it should be translated to a QJsonDocument and then  into a QObject. This QObject can be easily consumed by the GUI, which is written in the QML language.

An important future goal is extensibility. We will make frequent changes to our data model as the experiment progresses. We need to translate our model to repetitive code across several platforms while maintaining consistent names and relationships. Ideally, we would have one source-code for both the client and the server. Instead, we're using JSON objects, Qt objects (defined using C++), and Java objects to store and transmit information. To help keep the data objects consistent across platforms, we are working on a script that generates (1) a SQL create script, (2)Qt Object and JSON conversion code in C++, and (1) Java object declarations in order to keep names and types consistent and easy to update.

Update 02/19/17

Work Accomplished:
We have completed the IRB forms required for our application.

Goal:
Test the student application on iOS and complete the Protecting Human Subject Research Participants course, which is required by federal law for projects involving human subjects research.


Outcome:
I learned the requirements for getting the Institutional Review Board's approval for research involving human subjects.

--Eugene

Update 2/12/17

Work accomplished:
We have shortened the proceedings file and submitted the final version of our paper. After submitting the paper, we managed to complete the presentation for the SITE 2017 conference.

Goal:
We want to complete the necessary IRB forms and related documents in order to submit a proposal for the user study we will conduct.

Outcome:
I have gained an understanding of what professional research is, and why collaboration skills are important in projects like this.


--Eugene

Update 02/05/17

Work Accomplished:
Our research paper has been accepted to the SITE conference, which will take place March 5 - March 9.

Goal:
Shorten and finalize the proceedings file, along with create the presentation for the conference. I will also be making other plans in preparation for the conference.

Outcome:
By working on this paper, I've learned all the protocol involved in writing and submitting a research paper.

--Eugene

Update 1/29/17

Work accomplished:
We have completed a basic pattern generation script along with a basic patten entry method in the application. Further development in these two components will increase usability in the survey.  The client networking code has been rewritten using C++, which has eliminated the app-crashing bug on the Android platform.

Goal:
One of the major goals we have for the next few weeks is to complete all the required documentation for our IRB proposal. We expect the approval process to take 3-4 weeks, so it is crucial that we complete the proposal material as soon as possible. We will also continue working on the client-side and server-side components in an effort to finalize our system before the user study. The account creation aspect of our system will need to be implemented so that new users can log in to the system.

Outcome:
We will have permission from the Institutional Review Board to collect data from testing our application in classrooms.

Update 1/23/17

We have completed the collaborative research paper and we have submitted the paper to the organization. We have managed to get QR functionality in the app by using the quirc QR decoder library. We have yet to get the in-app camera working. In the coming weeks we will continue developing the in-app pattern submission system, along with server-side pattern generation.

--Eugene

Update 1/15/17

In the past few weeks there has been a lot of focus on the collaborative research paper that we will be submitting to the Association for the Advancement of Computing in Education's SITE conference. We are on schedule to complete the full paper that details the overall system of our project, along with some information regarding the evaluation plan for our experiment. We have been rewriting some of the code used for communication between the network and server since there were some bugs created by the JavaScript code. We will also be modifying the application's user interface to better resemble what the application should look like in its completed state.

--Eugene