CS10 : Hear, hear!

Apart from images, text and numbers, computers can also store sound. Not in analogue format like we hear it but in digital format. How does this work? How is an analogue wave converted into a digital signal? Fascinating.

We are learning ...
• About the ways in which sound can be stored and represented in computer systems
So that we can ...
• Describe the principles of storage of sound waves using sampling techniques
• State the difference between analogue and digital data
• Describe how the ADC works and its application to sound sampling
• Describe how the DAC works and its application to sound reproduction
• Understand the concepts of sampling rate and sampling resolution
• Calculate the size of a sound sample
• Calculate the size of a raw sound file
• Describe the purpose of MIDI and how it works
• Describe the advantages and disadvantages of using MIDI to represent music.

It is also very likely that you might need some headphones as well!

 Activity 1 Binary data (10)

Since computers are made solely from very tiny switches which can only be on or off, the computer can only store data in binary format. Binary data inside a computer can represent one of many different things such as an integer number.

 Task 1.1 What could it be? Whiteboards and pens Look carefully at the following portion of binary data ... Click to enlarge OUTCOME : Write on your whiteboards what this portion of binary data could represent.

No checkpoint for this

 Activity 2 What is sound? (35)

When we hear or produce sound, it is in the form of an analogue, longitudinal wave. Analogue is continuously variable, not discrete like digital data so computers find it difficult to handle. Watch the video and then try the task which follows.

How do your ears work? (5:08)

 Task 2.1 The dynamic microphone None Investigate the operation of the simple dynamic microphone. You may wish to watch the following YouTube video to help you and perform a google image search to find pictures of a dynamic microphone. How do microphones work? (1:00) OUTCOME : A simple diagram for your notes explaining how the dynamic microphone works, including a definition of the word 'analogue' in your diagram.

 Activity 3 The analogue to digital converter (30)

Sound is transmitted as a series of pressure differences in a medium like the air. For the computer to be able to store these pressure differences in a way which will allow it to accurately represent the sound, the analogue signals need to be converted into digital signals.

The ADC converts analogue signals into digital data

 Task 3.1 The ADC (and the DAC). Notebooks Simply copy the following diagram into your notebooks - easy. Click to enlarge OUTCOME : A beautiful diagram showing ADC / Computer / DAC in your notebooks.

The analogue to digital converter (ADC) is responsible for converting the analogue sound into digital data.  It's important that, now we've seen the ADC in context, that we understand how it works. Try the following task.

 Task 3.2 The operation of the ADC. Notebooks Once again, YouTube jumps to our rescue on this one. There is quite a nice practical demonstration of an Analogue To Digital convert in this video. Listen carefully for the 'comparator' ... How does an ADC work? (3:58) Now take some notes on the operation of the ADC in your notebooks. Include diagrams if necessary. OUTCOME : Notes and diagrams in your books describing how an ADC works, and maybe even a practical example of one you have built yourself?

 Activity 4 The Digital to Analogue converter (30) A Level Only

So we've looked at one side of the conversion - the analogue to digital convertor. What about the other side?

Once again, YouTube explains this much better than I could! Watch these ...

Collin's Lab: Digital to Analog Converter (4:56)

Electronic Basics #10: Digital to Analog Converter (DAC). Listen at 3:53 :) (5:55)

 Task 4.1 The DAC Notebooks After watching the two YouTube videos, use diagrams and a written explanation to discuss the operation of a R-2R Digital to Analogue Converter. If you are feeling 'makey', follow the link to buy yourself an Arduino Genuino Micro (the Arduino Nano is 'retired' now). OUTCOME : Description of operation of a Digital to Analogue Converter (DAC).

 Activity 5 Investigating a sound (85)

Sound waves are represented by transverse waves in the computers memory. The displacement of the wave describes the absolute pressure change and the frequency of the wave describes the pitch of the sound.

Warning
If you listen to loud sounds through your earbuds at high volume, you will damage your hearing.

 Task 5.1 Investigating sine waves Audacity 16Hz to 20kHz (CA).mp3 16Hz to 20kHz (DA).mp3 There are two sound files for you to investigate in this activity.  Download and open each sound in Audacity. Firstly, investigate the 16Hz to 20kHz (CA).mp3 sine wave. CA stands for 'constant amplitude'.  Notice how as the sound increases in pitch, the waves get closer together (the wavelength reduces / frequency increases).  The amplitude of the wave remains constant, however, since the acoustic power increases as the square of frequency, it may seem as though the sound gets louder as the frequency increases.  The sound is 10 seconds long.  How long is it until you can't hear any more?  Write about what you have found out in your notebooks (using screenshots if necessary). Next, investigate the 16Hz to 20kHz (DA).mp3 sine wave.  DA stands for 'decreasing amplitude'. When you play this sound, the loudness of the sound should remain constant even though the amplitude of the wave decreases.  Write about what you have found out in your notebooks (using screenshots if necessary). You will also notice that if you zoom into the file (CTRL + Mouse Wheel) and scan from low frequency (left) to high frequency (right), the waveform gets less 'regular' and more 'jagged'.  Why do you think this is? OUTCOME : Notes and diagrams in your notebooks documenting your investigation of sine waves.

Real sounds are an extremely complex mix of multiple harmonics.  Watch the following YouTube video which will help you to understand how complex real sound is; hopefully, it'll stop you asking me questions!

Complex sounds : Sound theory (5:08)

 Task 5.2 Investigating a sound Audacity Drum beats.mp3 Download and open the Drum beats.mp3 sound file using Audacity. You should firstly inspect the sound file - a single wave to represent all the richness of the music, amazing.  Zoom right into an 'exciting' part of the file (with lots of ups and downs).  You should reach a point where you see, at regular intervals, small dots on the wave.  Now choose the pencil tool from the toolbar and investigate what you can do and how / whether the sound changes as a result. Click to enlarge Next, starting with a fresh copy of the Drum beats file, investigate the envelope tool.  Again, what does it do and how does it effect the sound? Click to enlarge Lastly, try applying some effects to a fresh copy of the Drum beats file.  There are loads of effects to choose from and you can apply them to a small part of the file with the selection tool (the 'I-beam'). OUTCOME : A write up of your investigation of a sound file including screenshots and sketches in your explanations.

 Activity 6 Sound sampling (65)

As you have seen from the investigation of the sound waves in Audacity, they are stored in a digital format as 'points' on the wave. But how do we work out where the points are?

Watch the two YouTube videos which explain much clearer than I could how sound sampling works.

Digital Audio 101 - Bit Depth, Sampling Rate, Interpolation (4:34)

Digital Audio 102 - PCM, Bit-Rate, Quantisation, Dithering, Nyquist's Sampling Theorem (6:06)

In summary (and you should take notes on this!) ...
• Accurate samples of analogue signal taken at regular intervals (sampling frequency / sampling rate).
• Must be at least twice highest frequency in the analogue signal (Nyquist Theorem).
• Gives a PAM trace (Pulse Amplitude Modulation).
• PAM data is then quantised by rounding up or down using a particular sampling resolution to produce a Pulse Amplitude Modulation (PAM) trace. 'n bit' sampling resolution results in 2n values available for this quantisation process.
• The quantised PAM trace (Pulse Code Modulation or PCM) encoded using n bits.
• Output from PCM encoder is a sequence of fixed height pulses representing the binary encoding.

 Task 6.1 Practical sound sampling Sound sampling exercise.docx Download the worksheet on sound sampling and carefully follow the instructions. Harry Nyquist once said ... "Sound must be sampled at at least twice its highest analogue frequency in order to extract all of the information from the bandwidth and accurately represent the original acoustic energy." So, why are CDs normally sampled at 44,100Hz? OUTCOME : Completed worksheet on sound sampling plus an answer to the Nyquist question!

These activities are carried out by the ADC - the analogue to digital converter - that we met earlier - in hardware. Every analogue sample is passed through the ADC in the manner which we saw in the video earlier and converted into a binary string. The sampling resolution is directly related to the number of positions in the DAC part which contained the resistors and LEDs in the video - clear?

 Activity 7 Calculating the size of a sound file (20)

From a knowledge of the sampling resolution, the sampling rate and the length of the sound file, it is possible to work out approximately what storage will be required for a sound file.

For example ...

Sampling resolution : 16 bits = 2 bytes
Sampling rate : 44000Hz
Length of sound : 3 minutes = 180 seconds

Approximate file size (in bytes) = 2 x 44000 x 180 = 15,840,000 bytes = 15.1MB

This file size is uncompressed - the size of a `.wav` file.

 Task 7.1 Calculating file sizes None Calculate the approximate, appropriate, uncompressed file size for the following sound waves : 16 bit sampling resolution, 16000Hz sampling rate, 12 seconds long 8 bit sampling resolution, 8kHz sampling rate, 1m 5s long 24 bit sampling resolution, 44.1kHz sampling rate, 4m 28s long OUTCOME : Written solution to problems involving files sizes of sound files

 Activity 8 Sound synthesis (75)

Sound synthesis involves the production of sound waves which represent or sound like real world sounds but were not sampled from analogue waves.

MIDI - Musical Instrument Digital Interface

Speech synthesis

Professor Steven Hawking would find it difficult to communicate if it wasn't for speech synthesis. There are three main methods of reproduction of the human voice ...
• Concatenative synthesis : speech is made from the concatenation of segments of recorded speech;
• Formant synthesis : where the speech is constructed using algorithms;
• Articulatory synthesis : where computational or physical models of the human vocal tract produce speech.
So, first the wonderful 'Texas Instruments Speak and Spell' ...

Texas Instruments Speak and Spell (2:32)

Next, one of my all time favourite games (showing my age) - Tales of the Arabian Nights on the C64 ...

Tales of the Arabian Nights (1:08)

Finally, the car everyone wanted - the Austin Maestro Vanden Plas featuring the voice of Nicolette Mackenzie ...

Austin Maestro Talking Dashboard (0:32)

 Task 8.2 TED Talk Nothing required This is another great TED talk. Enjoy. TED Video - Synthetic Voices (11:40) OUTCOME : A simple review of the TED video in your notebooks.

 Extension Activities