University Office Hour Helper - the COVID Version

Social distancing creates many challenges that we need to deal with nowadays. However, it also creates opportunities that we can benefit from if we manage to take advantage of them. The following project uses the benefits of the situation forced upon us to make things easier for University instructors, professors and the students themselves. It aims to assist in running the office hour that many instructors and professors offer in remote more efficiently and organized, both for them and for the students.

The fact that many sessions at universities all over the world are now remote brings with it advantages and disadvantages. The use of appropriate technology maximizes the benefits of remote sessions while minimizing the problems caused by not being in the same room together.

This project was built as part of a computer science course at Reichman University, Israel.

The hardware required is a circuit playground express board and a ESP8266 shield for connectivity.

The circuit playground express board contains the following:

• 10 x mini NeoPixels, each one can display any color
• 1 x Motion sensor (LIS3DH triple-axis accelerometer with tap detection, free-fall detection)
• 1 x Temperature sensor (thermistor)
• 1 x Light sensor (phototransistor).
• 1 x Sound sensor (MEMS microphone)
• 1 x Mini speaker with class D amplifier (7.5mm magnetic speaker/buzzer)
• 2 x Push buttons, labeled A and B
• 1 x Slide switch
• Infrared receiver and transmitter.
• 8 x alligator-clip friendly input/output pins
• Includes I2C, UART, 8 pins that can do analog inputs, multiple PWM output
• 7 pads can act as capacitive touch inputs and the 1 remaining is a true analog output
• Green "ON" LED so you know its powered
• Red "#13" LED for basic blinking
• Reset button
• ATSAMD21 ARM Cortex M0 Processor, running at 3.3V and 48MHz
• 2 MB of SPI Flash storage.
• MicroUSB port for programming and debugging
• USB port can act like serial port, keyboard, mouse, joystick or MIDI...

For this project we used the following elements of the board:

1. Neopixel leds.
2. Light sensor.
3. Motion sensor.
4. Speaker.
5. Push buttons.
6. Sound sensor.

In addition to what the board offers we used an ESP8266 shield to enable us to connect to the internet allowing us to use the Blynk app, the Integromat service and an MQTT service called adafruit.io.

The ESP8266 shield is connected to the circuit playground express board via UART and is describes in the attached table.

The project is divided into two parts: the instructor side and the student side and its flow can be summarized in the attached flow chart.

The instructor code uses Blynk together with the interrogator and MQTT to send and receive messages to the students before and during the office hour.

The students receive messages from the instructor using an MQTT service. They can send messages to the instructor during the office hour (for example, hand clapping).

Technologies

Blynk

Blynk is a product that can control hardware remotely, display sensor data, store data, visualize it and more.

Blynk includes the Blynk App which enables us to create an easy to use interface that controls hardware or sends data to and from the hardware.

We used the blynk app in order to create an easy to use dashboard for the instructor to use during the office hour.

In addition, we used the Blynk webhooks feature in order to use the Integromat and Adafruit.IO services which will be described below.

Integromat

The Integromat service is a web service that allows us to integrate and automate different services into one flow.

It includes integrations to the services google, email, web requests, social network services and many more.

We used integromat to automate some flows that included transferring data between different services and manipulating this data.

Adafruit.IO

The Adafruit.IO is a cloud service that allows us to use MQTT for communication between the instructor and the students.

MQTT is a publish-subscribe network protocol that transports messages between devices.

The information on Adafruit.IO is transferred using feeds and can be visualized by creating a dashboard which is a feature that Adafruit.IO offers.

As mentioned before, we used the Blynk app to build an easy to use dashboard for the instructor.

We created a device for the instructor and added a mobile and web dashboard which looks like the second picture attached at the beginning of this step.

The Slider widget is used to send the currently reviewed question and the Button widget will be used to send a message to the students letting them know when the office hour is about to start.

We used the data streams presented in the above picture for input and output to and from Blynk.

We created 4 datastreams:

1. V0 - Virtual pin datastream of type String which will be used to send webhooks to the Integromat using the Blynk webhook feature.
2. V1 - Virtual pin datastream of type Integer which will be used to send the currently reviewed question.
3. V2 - Virtual pin datastream of type Integer which will be used to send the students a message when the office hour starts.
4. V3 - Virtual pin datastream of type Integer which will be used to send to send the Instructor messages from the students.

Blynk also includes a webhook feature which enabled us to use it to send a webhook request to the Integromat service as will be presented below.

Adafruit.IO is used as an MQTT service to transfer messages between the instructor and the students.

We created 3 feeds where each feed is used to transfer different types of messages:

1. Questions Feed - The instructor will publish to this feed which question he is about to review.
2. Instructor Feed - The instructor will publish different messages on this feed to the students. For instance, if there is a break or to notify them that the office hour is about to start.
3. Students Feed - The students will publish messages they want to send the instructor on this feed.

This step will show how we arranged the emails received from the students and decided which questions will be reviewed in the office hour.

For this step we used the Integromat service which helped us automate this process and integrate our email service with google sheets and with blynk and adafruit.io.

For the collection and arrangement of the emails we created the scenario that is presented at the beginning of this step.

This scenario will run every 15 minutes and each time will collect all of the emails from the inbox that haven't been read yet and that have the keyword "homework" in their subject.

Then it will write the information from the emails into the google sheets spreadsheet. Eventually, every row in the spreadsheet will contain information from one email: date, question and sender email.

The google sheets spreadsheet will eventually look similar to the second attached picture at the head of this step and can be viewed here.

This requires that the emails that the students send contain the keyword "homework" in their subject and that the email body contains nothing but the question.

The above picture is the next Integromat scenario to be executed.

It will be executed by the instructor just before the office hour starts and is responsible to go over the collected questions and check which ones should be reviewed and in what order. Then it will notify both the instructor and the students about the result via email.

The scenario will be executed by the instructor by clicking the left button on the circuit.

As can be seen, at the end of the scenario, in addition to sending the results we also notify the instructor that the scenario is done on the Circuit Playground express by sending a message to the virtual pin V3 over Blynk. We use the Blynk REST API for that and the request is as follows:

https://blynk.cloud/external/api/batch/update?token=<your_token>&V3=1

In addition, it is important to note that this scenerio is triggered by a webhook that is recieved from Blynk.

We setup the Circuit Playground express, so that when the instructor will press the left button, a request will be sent to blynk which will send a request to the Integromat.

We used the Blynk to send the webhook requests to Integromat because our ESP8266 component had a problem with sending HTTPS requests, but if you prefer and are able to you can send the request directly from your circuit.

Because we couldn't handle both Blynk and MQTT connections on our circuit at the same time we had to use Blynk to send and receive messages to feeds on Adafruit.IO.

For it to work we used the Blynk webhook feature again with different parameters to differentiate the different scenarios we wanted to execute on the Integromat using the same webhook.

The scenarios on Integromat enabled us to send messages to and from the instructor over Adafruit.IO without running an MQTT client on the instructor circuit.

To send and receive data over the Adafruit.IO feeds we used the Adafruit.IO API service which is well documented here.

We used one big Integromat scenario for all of the messages the instructor can send the students. It is presented at the beginning of this step.

This scenario is triggered by a webhook request from the instructor which can contain one of the following parameters (using the filter feature of Integromat):

1. quest - used when the instructor sends the number of the next question to be reviewed to the students. In this scenario the message is sent to the all students "Instructor" feeds and an email is sent to the students that asked this question. We get the students email addresses from the google sheet we saved by querying for rows of the people that asked the specific question.
2. start - used when the instructor sends a message to the students that the office hour is about to start. This message is also sent over the "Instructor" feeds.
3. break - used when the instructor wants to tell the students there is a break. This message is also sent over the "Instructor" feeds.
4. over - used when the instructor wants to tell the students that the break is over. This message is also sent over the "Instructor" feeds.

The integromat scenario will be set to run periodically.

We created another scenario for the messages that the students can send the instructor.

Because of the MQTT limitation that was described above, we used the virtual pin V3 on the instructor Blynk to receive messages from the students feed which will be used by the students to send messages to the instructor.

To send messages to Blynk, we used the following request:

https://blynk.cloud/external/api/batch/update?token==<your_token>&V3=<value_from_instructor>

The scenario (which is presented at the beginning of this step) is executed periodically every 15 minutes and will make a request to the Adafruit.IO API service:

https://io.adafruit.com/api/v2/<username>/feeds/office-hour.students/data/last

This request will ask for the last data that was received on the feed, and in order to make sure this is new data we filter the response and check that the data was received in the last 15 minutes.

Our code is separated into 2 parts, one that will run on the instructor circuit and one that will run on the students circuits.

Both sketches are attached at the end of this step.

Dependencies

The code needs to use the following libraries which should be installed before loading the code:

1. PubSubClient
2. TinyGSM
3. ESP8266WiFi
4. Adafruit_NeoPixel
5. Blynk
6. BlynkESP8266
7. PItches header file

The Instructor Code

The sketch will execute the Blynk client and will use it to send and receive messages. It will use the buttons and light sensor on the circuit as input and the pixel lights as output on the circuit.

The messages to the instructor are received by using the virtual pin V3 on Blynk and include:

1. V3 = 1 - indicates that the email check process is done and the instructor can start the office hour.
2. V3 = 2 - indicates that the instructor received hand claps from the students.

The messages that the instructor sends are sent from the Blynk dashboard or by using the buttons and light sensor on the circuit and include:

1. Left button press will start the email check process on Integromat which is described in step 6.
2. A low light sensor value will send a message to the students that there is a break. During the break the light sensor value is monitored and when it is high enough it will send a message to the students that the break is over.
3. Blynk input messages from the virtual pins V1 and V2 which send messages to the students are listed in step 3.

The Students Code

The sketch will execute an MQTT client that will connect to Adafruit.IO. It will use the sound sensor as input and the neopixel lights and microphone as output on the circuit.

The messages to the students are sent over the feeds "office-hour.instructor" and "office-hour.questions" and are processed in the function "mqttCallback".

The messages that the students send to the instructor are sent on the "office-hour.students" feed and are sent according to the input from the circuit such as the sound sensor.

Parameters

For both sketches you will need to enter your wifi information.

In the instructor code you will need to enter you blynk device details which can be found on the Blynk website in your account under the relevant device.

In the students code you will need to enter your adafruit.io details:

1. MQTT_STUDENT_USERNAME - student adafruit username.
2. MQTT_USERNAME -adafruit username.
3. MQTT_PASSWORD - the password can be found under the "My Key" tab on your Adafruit.IO account.