Media Technology course aims at presenting the knowledge-based foundations to the digital media undergraduates. It covers:

• Image, Video and Audio acquisition, processing and compression
• Fundamentals of Light and Sound
• Digital information storage and Content Delivery Networks (CDNs) In the BCU module guide, three learning objectives should be fulfilled, which are:
• Demonstrate an understanding of the operating characteristic and applications of systems involving capture and processing of Image, audio and human-interaction.
• Explain the application of communication and networking in media processing and distribution systems including: how audio-visual information is transmitted on the internet.
• Present original analysis of technologies and markets from wide ranging research.

This course is different from most of “design courses” for it requires students to write a logical essay rather than use the learned knowledge to create a work (Video, Audio, Website and so on). However, the report’s topic is decided by students who should address a specific area related to video compression.

The Challenge

The students struggled not only because the essay should be written in English, but also the way to “create” their own essay. Usually, “art-based” students are trained think and create a work(artwork) subjectively. However, they are confused by presenting “objective” knowledge and are required to write an essay to address their own thinking.

Trained as an Interaction Designer, I reminded how effective and efficient the “card sorting” method is when we designers try to learning new information under unfamiliar conditions.

Usually, the card sorting is conducted between two people (one interviewer and on interviewee). In my previous experience, I found the essential part of card sorting is by asking and speaking out the thinking inside the brains. In my teaching activities (tutorial classes), I generated several questions based on the lecture contents and asked students to search answers after classes. The students are encouraged to ask their own questions and will receive additional credits when they proposed good questions.

From “Knowledge” to “Questions”

The courses include lectures, workshops and independent study. I used the workshops to guide the students to practice the learned knowledge (for example, shown the “Wagon Wheel Effect” by let them shoot a rotating wheel). However, I also help them formulate the questions based on their lectures. Students should use independent study as their own time to answer these questions and propose more valuable questions.

Here is an example from “Week1: Foundations of Digital Camera” to show how I “transfer” knowledge into questions

Lecture Covered

1. Visible Light Spectrum
2. Additive(light) vs Subtractive(paint)
3. Blackbody Radiation and Color temperature
4. Digital image sensors
5. CMOS vs CCD
6. RAW format
7. Aliasing
8. Moiré pattern
9. Rolling Shutter
10. Wagon Wheel Effect
11. Partial Exposure

Questions assigned to Students

1. What is light?
2. What are the differences between RGB and CMYK?
3. What does a color filter do?
4. What are the different between Full Frame Camera and APS-C camera?
5. What is bit depth?
6. What are the differences between 8-bit and 16-bitWhat is crop factor?
7. What is crop factor?
8. Which one of CCDs and CMOS sensors is using “rolling shutter”?
9. Why there will be a wagon wheel effect?

Card Sorting for Writing

The final assessment asked students to use the knowledge of this course to generate an essay about “why compression is important”. This is the card note from one students about the question “what is bit depth”:

Bit depth is the number of different values which quantify the intensity of a pixel, for example, in color images, bit depth refers to the number of values for each pixel in each color channel. 8 bits = 256 (2^8) different values

Here is how she linked “bit depth” to the general topic “compression”:

In terms of image and video, the visual information in the physical world is captured by pixels, which is composed of bits and will be rendered by the computers. As the digital technology samples the physical signals into digital information, the main challenge (or problem) is to keep as much as “useful information” while constraints the file size affordable. From the business point of view, the digital video has to be kept as the enough quality and be as small as possible to keep in the server. Therefore, compression is one of the most important things in the process of capturing, storing and distributing videos.

In the course “Processing Art”, I introduce an artist or art style each week to inspire the students. In this week, We learned the “for” loop to represent the art style of Yayoi Kusama, who used a lot of strong colored dots to draw. Every week, I tried to combine artistic thinking into the basic knowledge of coding. In order to encourage the students to create, I introduced the “four-coordinate plane”, a method to research and organize information. Students could collect and place different artists or art styles into different planes, then try to modify their code to achieve different visual effects.

[Check the basic example on Github]

Student Works:

Find, Zhang Ruheng

Colorful, Long Ziyue

Mario Jump, Tong Xirui

mouseX, Zhao Tian

Final Works

Greasy Chin, Tong Xirui

First of all, the theme of this work is clear, it is a very greasy uncle’s greasy chin and there are many yellow and red acne on his chin.Because it is my own drawing, this greasy chin is not so disgusting, and actually it is a little cute, round nose and round chin, vivid. The user could click and drag the chin, and the chin can be deformed accordingly, and the chin can be dragged at will. This is a bit funny and interactive. Because my technique is not enough, the acne will fly out of the chin, so I made an air wall. The mouse can’t drag. I can only do this. Hahahaha. What do I want to say in this work? That is, we can’t become too greasy. Otherwise, the chin will not be well maintained and it will be distorted.

Scary Nostril, Tu Qingying

I got the theme with the word “Scary”, and naturally I thought that the visual style should be a bit scary. And at the beginning, nobody could see the specific photo inside the black background, and with the clicks the scary picture blows up, it can have a strong effect after the image is enlarged. Click on any key on the keyboard to restore.

Lazy Machine, Long Ziyue

My theme is lazy mechanics, so I drew a small rocket, slow and slow to show that it was lazy. In order to make the picture more vivid, I added a little interactive effect: when the mouse moves on the screen, there will be a small ball to follow. That’s it………

Silly TNP, Zhao Tian

The theme is … emm “Silly TNP(Taiwan National Party)”! When I got the theme, I am actually very desperate and confused… Later I thought of combining the TNP elements with the funny emoji. The interactive effect of the rotation is mainly to fully reflect the characteristics of “silly”. When the mouse moves to different places, there will be different speeds just to make the content more interesting and interesting…

Shamate Meat, Wang Chenziwei

Theme: Shamate (meaning extremely out-of-fashion in China) Meat Visual Style: I want to make it as Shamate as possbile, but it seems that it is a little cute than Shamate Interaction effects: control the size of the eyes with the mouse movements

Sissy Spade, Ren Mengying

“Do not live in the framework of society. Femininity is not necessarily bad, masculine is not necessarily what you want to pursue. No matter male or female, there is no established framework for gender identity.” My theme is “Sissy Spade”. Because I want to reflect the sense of “sissy”, the background is a colorful gradient. I want to break the static and add visual diversity. So the unhappy spade in the work are shaking with the background music, and various labels for the “sissy boy” rise from below.

This work is based on the general perception towards “sissy boy” in the society. To face of the group of “sissy spades” that is limited by the society, I hope that there will be less ridicule or fanaticism. A spade is a inverted heart, with my expectation of respecting the diversity of the society.

Retarded Melon, Sang Mengshen

Free Mr.Banana, Wu Xinyu

Because the general colors of the work make people want to dance (perhaps not) when the flashing Mr.Banana is accompanied by such colors, it makes people feel his freedom (well, it’s just a bullshit).

Interaction effect: Make you want to dance when you look at it.

Peaceful Me, He Wencheng

It is very calm, without any movements and sounds, always watching the sun in the sky until the night and it is dark, he is still looking to the sky, but unfortunately can not wait for a round of moon, it is thinking of why is there no moon? The sun is also very calm, alternating every day in a constant dimension, and in this way, an infinite loop.

Terrible Nut, Zhang Hanjun

Why choose the current visual style: Because it looks short and sudden, you can scare the timid person like me. I used the “JUMP SCARED” that I liked in the American horror movies.

The interaction effect: Move the mouse on the black screen, and you can harvest a cute and touching nut at a specific location.

Mojie Tang, Yuqing Zhou, Zimeng Huang

The project was inspired by a stomach ailment, which was tortured by stomach pain and understood the importance of the stomach. The stomach is not just a digestive container, it is also a friend who has been with us for a lifetime.

We use Processing to make a stomach shape, render a pink background, fill it with green particles that symbolize gastric juice, and press the middle button to add a yellow “drug”; press the mouse wheel to place the stomach drug for treatment The effect of pressing shift + left mouse button can be used to launch the bullet to stir the stomach juice; the right button can eliminate the gastric juice.

The audience finds a kind of fun in the interaction, which leads to an inspiration: whether we know our stomach and whether we can treat our stomach well.

[View the source code on Github]

Synesthesia Alphabet(通感症字母表)

Ruihan Yu, Ying Huang, Wenqian Hu, Yiming Wuhuang

What is the world of the people with synesthesia? Every word, every sentence, has a unique color, smell and sound. Words are like water, which are the basic element: in the sky, they are flowing clouds; on the earth, they are falling water. When they are arranged in a special way, they will shine like the rain and moisten the world in our hearts. When we look at the world with different eyes, those beautiful words will linger in our hearts.

[View the source code on Github]

The Choice Before And Now(过去和现在的选择)

Zehuan Xu, Shiru Wang

Our project concept originally comes from Dali’s painting named “The Persistence of Memory”, which is related to the clock (time). We wanted to express the things in our life and they gradually took up the time of our lives. But this idea has caused us a lot of troubles - too many things to choose.

So we shortened the time to three years in high school and changed it to a questionnaire. When the audiences face each item, they can click on the color they think to represent the period.

We want to collect people’s memories and thoughts about high school and make a work similar to data visualization. Now, think about the choices made at the beginning, are they really what we thought before?

[View the source code on Github]

Corpulence(吃胖)

Jingtong Cheng, Qiuyu Yu, Qian He, Junyue Zhang

“Obesity” is linked to un-beauty and squatting. It is a common group of metabolic syndrome. The volume and amount of fat cells in the body increase abnormally, and the percentage of body fat to body weight is abnormally increased. When the body eats more calories than consumption, fat synthesis, and it is the material basis of obesity.

Usually, the girls think that “only food and love can not be disappointed, spring is too short, it is better to eat fat”. So sweet or hot and crisp high-calorie food are added to their stomaches. A unit of calories can raise one gram of water by one degree Celsius at a standard atmospheric pressure. As an energy unit, it is almost same as heat. Calories come from carbohydrates, fats, and proteins, and these three things come from food.

The calorie value represents a possibility of a human-balanced relationship with food, but it is far more than warning of fat. The other side of the ease is often the hidden dangers; another possibility comes from the turn-away from donuts, the cheese-flavored chicken, and then: to enter the world of great inspiration for humanity.

[View the source code on Github]

Hot As Dog(热成狗)

Ruzhe Cai, Duanying ZHang, Hehui Zhang, Fei Xie

The inspiration for our interactive design is the summer days of Wuhan. As global warming becomes more severe and glaciers melt, global warming has threatened our environment. In face of this problem, we are caught in thinking.

In this design, we use temperature sensors to interact with human body temperature changes. At lower temperatures, the puppies and cows are in a normal state, and the temperature of the sensor rises with the holding time while holding the sensor. At the same time, the dog and the cow begin to melt and deform, and the icons on the display will change from puppies and cows to hot dogs and milk. When the temperature falls back, the opposite changing process will be presented.

Akerlof said: In the face of global warming, we lack a story of motivational action. And our interaction wants to use the form of temperature visualization to call people to participate in it and act.

[View the source code on Github]

Cellutopia(细胞社会)

Yinyin Yu, Wenwen Yang, Yiwen Liu

We believe that the living state of an ideal society is that people can fully realize their own ideas without affecting others. For example, you can decide to pursue an ideal job and decide your own life direction. However, this freedom is completely detached from the material basis and is actually difficult to achieve.

Real life is a combination of material and spiritual, and also a conflict between material and spiritual. Compared with an ideal freedom, reality presents a state of disorder and turbulence. This work is in opposition to pure spirit and uses material expression to reflect the spiritual oppression brought about by reality.

The protagonist of the work is the unit of matter that constitutes human beings - cells, each cell representing a person in the purely physical sense. The regular squares represent social rules.

The first part focuses on a person walking leisurely in an ideal society out of the limits. The second part reflects people’s nervous reaction to life through its jitter. It reflects the spiritual state of real life through material expression, contrasts with the first part, highlights the conflict and integration of ideal and reality, revealing subtle irony.

[View the source code on Github]