(4/28) A brief analysis of work completed

Over the course of the program, Hugo, Eugene and I have learned much more than I think any of us expected to about the research process, mobile development, databases, and creating a RESTful API server. This post will serve to summarize the work that we've done on the server, and where our research will go from here.

We have written 3845 lines of code for the server. About 92% of this code is Java, leaving about 8% of the code as HTML/CSS. We have made 201 commits to our master branch for the server, which represents about 1 commit per day. The graph of these commits shows a clear spike as we neared the start of the experiment, and another spike as we ran into bugs that prevented us from testing in early April. Despite these spikes, the graph also shows that we have been consistently working on the server since early October.

The CREU program ends on 5/5, which is the same day that Eugene and I will oversee the last session our app is used. Once the final session is complete, we will issue an exit survey, which asks students to rank the methods of authentication and asks questions about the usability of the application. After this, we will no longer have funding, but Dr. Zheng and I have arranged for me to take Directed Research course over the Summer to analyze the data we have collected from the experiment, write a second research paper, and to write any documentation necessary for a future CREU team to continue to use our application for their research. This CREU team could add notifications to our application to study the effect of personalized feedback on student attendance rates, or could move forward with authentication using facial recognition or location determination via Wi-Fi networks.

In all, I'm extremely grateful for the opportunity to have worked on this project. This is definitely the longest I have ever worked on a single software project, the largest codebase I have ever worked with, and the most demanding in terms of my time. I have learned much more than I thought I would, gotten a feel for the research process, and worked with some really amazing people.

--Nico

(4/23) Another hiccup

This is the fifth in a series of backdated blog posts.

Our application now works with all methods, though we did suffer another setback last week. We host the server component of our application on a virtual machine managed by NMT's Computer Science department, which had its storage machines go down Monday of this week. Because of this, many machines' hard drives went into read-only mode, including the machine hosting our server. Even though our machine was restarted, our Tomcat instance was not, and so no students were able to sign in Monday. We compensated for the delay by holding two sessions, so this didn't affect our schedule.

Our application is also in the review process for the Apple App Store, which means that, proctoring the last few sessions and performing data analysis aside, we've basically reached the end of our project. I'll make a post next week detailing the work done on the server side, and our continuing research after the end of this year's CREU program.

--Nico

(4/16) Preparing for the SRS

This is the fourth in a series of backdated blog posts.

Work on the server has largely been completed, so Eugene and I spent this week preparing our poster for the SRS using the LaTeX template that Hugo created for us. We've completed the test sessions for both PIN and Password, and will probably be conducting two more tests for PIN and Password to give Hugo time to resolve UI bugs caused by adding the QR method.

The SRS takes place on Thursday, April 20, and we will be presenting our poster to passers-by and judges. They will submit surveys indicating their understanding of our research and our clarity in presenting.

--Nico

(4/9) Hiccups in the experiment

This is the third in a series of backdated blog entries.

As Eugene has mentioned already, the experiment schedule is being pushed back a week.

Before Monday, the intended first day of the experiment, Hugo, Eugene and I worked hard to finalize the application. We generated user accounts, finalized the communication protocol between the client and server, and simplified the instructor's web application. However, I had relied on behavior that was previously in our communication protocol to determine the course that the user was in, and because I tested the login and attendance submission process using a REST client, was unaware that the application was not sending the course number that the students were in. As such, on Monday, no students were able to log in. The fix was relatively simple-- we could just look up the student's course number in the database, as intended.

On Wednesday, we encountered another bug. The client expected that the position of a student's course in the list sent  would also be the course's ID number, but this was true only for accounts enrolled in our test session. This caused everything to look like it was working as we were testing our changes, but when students enrolled in the actual classes logged in, they could not submit attendance to the correct class because it was greyed out.

We fixed these issues before Friday and conducted a successful test in one class's lab session, which was a welcome relief.

We also enabled TLS for both the client and server before Wednesday, which was a difficult process for Hugo and I. The solution on my end was to stop attempting to enable TLS using Tomcat, which is needlessly complex, and to instead use an NGINX reverse-proxy.

In all, both setbacks were frustrating and time-consuming to resolve, but didn't affect our experiment too much.

--Nico

(4/2) Beginning the experiment

This is the second in a series of backdated blog entries.

We began the experiment Monday, beginning with taking attendance traditionally. This is being done both to give students a point of reference for when we start to use our application, and to collect data on the time our application saves over traditional methods, if any.

Next week, Eugene and I will introduce the PIN and Password methods of authentication. The following week, we will introduce the QR and Pattern methods. The week following this, we will require participants to use PIN and Password unassisted, and the next week, require them to use QR and Pattern unassisted. The experiment is being structured this way so that we collect data from the unassisted sessions in order to eliminate the effect that unfamiliarity with the application may have on submitting attendance.

--Nico

(3/26) Return from conference and Spring Break

I have some catching up to do-- this is the first in a series of backdated posts detailing what I've done in the month since we returned from our conference.

Eugene's conference debriefing post contains most of what we did during SITE 2017, but he did mention that we had to take some time the day of our presentation to work more on our application.

As we were preparing to present, we needed to conduct a final test of our application to confirm everything was working correctly. Unfortunately, we were suddenly unable to record our attendance in the session even though everything had worked perfectly just two days prior. It took about 30 minutes to discover that we had been using the phone's local time to determine whether a session was open, or whether students were on time, late or absent-- this meant that with the shift to Austin's time zone, no class sessions were seen as available! This also could have allowed students who were late or absent to manipulate their phone's clock to appear as if they were on time, so we were glad to discover this bug, even at such an inconvenient time.

Once again, we're very grateful to CREU for funding our travel and to Dr. Kuo for accompanying us to SITE 2017.

--Nico

SRS Update

I adapted a template poster written in the LaTeX typesetting language so my teammates and I could present at the Student Research Symposium at our university.
I created an outline of our work and discussed the poster's contents with Dr. Zheng and Dr. Ku, and I adjusted the poster's typographic features in order to squeeze in as much content into our limited 48 by 36 inch poster as possible.
A notable feature is our use of microtype in order to improve the appearance of the text using several techniques. These are masterfully described by Khirevich here: khirevich.com/latex/microtype/

Our presentation was Thursday, April 20th. Like our oral presentation at the SITE conference in Austin, our poster presentation inspired several useful comments from the educators in our audience. A highly desired feature was the ability to use our app for quizzes.

- Hugo

What I've Learned About Qt

I'd like to share some thoughts regarding the Qt framework, which we used to develop our cross-platform application. It sports an amazing feature set: any features that have become standard in a grpahical user interface are easy to implement. For example, a slide-out drawer menu as seen in many mobile applications is an easy to add feature in Qt. 

The Qt framework is written in C++, a flexible, efficient, and cross-platform
language. Thanks to the Android NDK C++ compiler and Apple's Clang C++ compiler, our Android and iOS apps benefit from easily integrated external libraries that provide advanced features. This includes the QR decoding library quirc, which I modified in order to use it to decode images captured using Qt's camera interface. excluded from the Qt framework some examples:  quirc for Qr decoding and mbedtls for cryptographically secure communications

The biggest drawback to the Qt framework is its large size. Our Android users must download an 11MB APK file when they first install the application from the Google Play store. This size delays updates and slows down our development. It also increases the complexity of our final product, a detriment to any security evaluations or debugging sessions. Although I attempted to enable our compilers' dead-code elimination features, which would remove any unused components and therefore produce a much smaller installation file, this feature is impossible to perform automatically with today's technology. This is due to our use of the programming languages QML and Javascript, the compiler cannot easily prove that these dynamic languages will nevery require a given Qt feature. Therefore the installation file must include the entirety of the Qt framework.

C++ and QML are difficult languages with unfriendly syntax (compared to Python) and fragile type systems that cannot prevent as many errors as the more advanced and consistent type systems of other languages, such as Haskell, Coq, Ocaml, Rust, and Java. Unfortunately, our team had to deal with several runtime errors that could've been prevented by a more advanced language.

- Hugo

Development Update: Features We Didn't Deploy and Things We Didn't Do

Although these features received some development time, we were unable to release them in time for our user study.

1. Countdown timer that displayed how long users had before their class began and how long they had until they were marked tardy or absent.
2. A detailed attendance log viewer so users could see a thorough list of their attendance reports during our user study.
3. A more efficient data representation using Google's protobuf data format. This would enable our client and server to communicate more rapidly than by transmitting data using the JSON format.
4. A faster database, such as MariaDB, would've let our server save and load data at a much faster rate.
5. A constant integration system that reported any compilation failures once we uploaded either client-side or server-side code to our code versioning system.
6. Unit tests for the UI so we could guarantee that users had access to given functionality.

The first 3 features required more thorough debugging, but we did not have time to finish them. Features 3 and 4 were not necessary as our server responded fast enough, with a mean latency of 50ms and no waits longer than about half a second. If we had more users communicating with our server using either the Android app or the iOS app, we would have to implement several optimizations such as features 3 and 4.

The last two features were sorely missed as we occasionally deployed updates that lacked key features. Any professional software development team would have implemented these in order to ensure that users would not be exposed to undue feature regressions.

- Hugo

Visual Design Update (04/26)

I spent about a week designing the application's icon. It remains visible on various backgrounds and avoids use of color combinations that would prevent color-blind users from identifying our application. I also wrote a Python script that converts the vector-graphic into a raster graphic of several dimensions. This was necessary because Android apps must contain the application icon in several dimensions in order to display high resolution icons to the diverse set of devices. Android icon files must be 24 by 24 pixels to 512 by 512 pixels. iOS has similar size requirements.



The application's user interface benefitted from Google's Material Design principles. I modified early prototypes in order to keep UI elements close to each other, and to use depth in the 2D user interface in order to follow the Material Design's paper metaphor. For example, buttons appear higher than text input elements, and the menu drawer appears to cast a shadow from an even greater height. In order to improve accessibility, I implemented a font size changer. This was required because Qt ignores the system-wide font size setting. However, we neglected to use Qt's accesbilities features which would've made our program more readable to screen reading devices.

- Hugo

Our app on an Android app launcher.

The login screen

The slide-out menu.

The help screen.

Need to change your password?

A course list (with a single course)

Result of reporting attendance.

Taking a picture of a QR code to report attendance.

QR code detected!

Exiting the application

Development Update (04/26)

Our server is responsible for receiving and storing courses, users, attendance logs, and attendance sessions.

My code generator ensures that the client and server are operating on data using a consistent naming scheme: a "User" object on the client is the same as a "User" object on the server, and they are both written to and read from our SQLite database efficiently. I created the data objects used in the client and server and I also setup the  SQLite database code in the server. Nico and I designed these data objects through a series of discussions about what kind of data we would need to communicate.

I also improved server security by enabling certain software configurations that tend to improve the confidentiality and integrity of the information we must store. This is in conjunction with improved monitoring by using a log viewing tool. This lets us view a live stream of who is communicating with our server.

- Hugo

What Students See

Students participating in the user study for our attendance tracking system can download the application by visiting an opt-in URL that takes them to the Google Play Store. They download a 10 to 12 MB APK file and wait for the installation process to finish.


- user logs in, picks a course, sends the appropriate authentication method
- in case of connection error: try to show a diagnosable error message
- sends salted hash of password to the server
- receives course list (typically one course)
- user picks a course and sends the appropriate
- pattern, password, pin are all QML
- QR: quirc computer vision library, QML camera
- tried to process full image, instead use C++ to access and process preview image
- QR croppableimage in QML
- compile and upload to play store (we don't have CI)

- Hugo

Security Update (04/26)

Our first time users must read the privacy policy before they can use the app. They are required to answer some short questions in order to demonstrate knowledge of the data that we require for our user study and of the data that we exclude from communications with our server.

1. we only collect anonymous research IDs, only instructors can correlate attendance logs with actual student names
2. we do not continously collect information, students send us information only when they click one of two specific buttons
3. we never transmit images through the network; QR codes within the image are decoded and the payload of the QR code is sent instead of the image itself

We use transport layer security (TLS, footnote: this used to be done with a now-obsolete technology known as SSL) in order to preserve data confidentiality (prevent snooping) as our application communicates with our server. This means that anyone "listening in" on the user's network traffic would only see a stream of random data, which could only be decrypted by our server.

The Qt networking library (We use networking libraries in order to send and receive data from our server.) helps us achieve encrypted communication on the iOS and MacOS platforms and the mbedtls networking library does the same on Linux and Android. Ideally, we would've used the Qt networking library for all of our platforms, but incompatibilities between Qt and Android prevented us from doing so.

My debugging sessions led me to discover that Android would load the of OpenSSL networking library present on the Android device instead of the OpenSSL networking library that we packaged for the Android application. This means that Android would prefer to use the software located on the user's phone (these are wildly different networking libraries), as opposed to loading a version of software that would be compatible with the Qt networking library.

So mbedtls, a cross-platform TLS networking library, became our Android and Linux networking library. Due to difficulties with the iOS build system, we continued to use the Qt networking library on Apple devices.

- Hugo

Development Update (04/26)

We have deployed and continue to maintain a complex program with about 40
users. Although data collection for our user study is still ongoing, this project taught us many valuable lessons regarding software engineering, user interfaces, and most importantly: communication.

The following blog posts are a report of the code I have written for this research project. We continue using the Qt framework to write a cross-platform application in the C++, QML, and Javascript programming languages. Our attendance taking application consists of about 1,800 lines of C++ code that is used for efficient networking, cryptography, QR code decoding, and other data processing; and about 2,700 lines of QML and Javascript code for describing the graphical layout and interactive elements of our app.

We share our code by uploading to our school's gitlab code versioning server and we discuss issues by using the Riot messenger and our school's issue tracking service.

- Hugo

Development Update: 03/16

We have been performing stress tests on our server. Eugene and I designed and implemented a testing script that starts 20 parallel threads that send various requests to the server as fast as possible. The tester sends correct and incorrect information, such as login requests and dummy attendance reports. The testing data is randomly generated --- for example, the script sends extremely large payloads (1MB) or valid and invalid usernames.
We have run the test concurrently on at most 4 machines (80 total connections). Our server has a response time of 100ms with a standard deviation of 50ms and max/min response times of 1000ms and 7ms. We'll use our newly developed stress testing script to evaluate several upcoming server-side changes and optimizations.

- Hugo

Update 4/23/17

Work accomplished:
Our group's poster presentation at the Student Research Symposium was a success!

Our group with our poster

Explaining our project to a group of our peers

Presenting to a NMT professor

Our project received a lot of positive feedback. One thing instructors would like to see in the application is a quiz authentication method -- where the instructor can ask a question and receive a response as an evidence type. Another instructor also suggested some form of integration with the iClicker system.

Goal:

Begin the unguided attendance sessions.

Outcome:

From these unguided sessions, we will ultimately gain an understanding of how the users feel about the system in a real-world situation.

-- Eugene

Update 4/16/17

Work accomplished:
Nico and I have demonstrated the PIN and password authentication methods. We allowed the study participants to ask any questions they might've had regarding the two methods. Our group has also completed the poster we will present at the SRS.

Goal:
In this next week we will demonstrate the two remaining methods. We will also be at the SRS on 4/20 from 11:20 - 12:30.

Outcome:
The participants will be familiar with the pattern and QR authentication methods. We will strengthen our presenting skills from our presentation at the SRS.

-- Eugene

Update 4/9/17

Work accomplished:
I have created a password generation script that will create random passwords for each study participant. The participants will be able to change their password once they have logged into the application. I have also wrote some code that inserted the user/password combinations into our database. We attempted to use the application on Monday, however there were some underlying issues that rendered the app unusable. After we thought all issues had been fixed, we attempted to test the application again on Wednesday with no luck. The server-side and client-side issues have been resolved and we had a successful test session on Friday.

Side note -- Our paper is now available here on the learntechlib website!

Goal:
Nico and I will demonstrate the PIN and password methods to the classes.

Outcome:
The participants will have a better idea of how to use the system and they will have the opportunity to ask any questions they might have.

-- Eugene

Update 4/2/17

Work accomplished:
We started the experiment on Monday and we began collecting data. For the first week of the experiment the instructors will use traditional attendance-taking methods.  One of the classes used a "quiz" method -- the instructor hands out a piece of paper with questions on it at the beginning of class, and collects them at the end of class. The other class used an attendance sheet method, where a piece of paper is passed around and students write their name. The data we collected this past week looks promising.

Goal:
This week we will begin using our system. I will also begin working on the poster that we will present at New Mexico Tech's Student Research Symposium (SRS).

Outcome:
I have learned what it is like to be a researcher in a classroom environment.

-- Eugene