Data floods into computer systems. It takes many forms; well, not that many actually. In this section, we take a look at the 'appearance' of data, how is is structured and how it is stored. A brief introduction to databases and file handling.

We are learning ...


So that we can ...



With the help of computers, humans organise raw data to give it meaning. Consider the following diagram. You may wish to sketch this in your notes. Where do we operate and where do computers operate? Give examples to illustrate your answer based on what you have discussed in class.


Data is raw facts and figures without structure, context or meaning...
Information is data with structure, context and meaning...


Knowledge involves awareness of the implications of the information developed from experience...
Wisdom is the power to make decisions based on the knowledge.





Big data is a broad term for data sets so large or complex and often unstructured that traditional data processing applications are inadequate. New methods of analysing large datasets have had to be developed in order to identify trends and patterns which help business and research to gain insight into the behaviour of populations who are generating data at an unprecedented rate.




Big data is created all the time in many different areas of life, for example ...





Computers need to categorise data on a fundamental level. There are 8 (or so) different fundamental datatypes which computers generally need to be able to deal with ...





Variable and constants (an introduction to terminology)



Naming variables and constants


Variable scope


Typecasting





At this stage, you need to be aware of the existence of the following different data structures. You will have lots of practice creating and managing these during the rest of the course - don't worry!

Before you start this section, create a blank word processed document with a suitable header and footer. Use screenshots and written explanation to document what you have done and what you have found out.




Open up the Python programming environment, IDLE.


Practice creating a simple list of data and accessing items in that list using their index. Type the following commands, pressing the Enter key after each one, twice at the end.

>>> myList = [12, 3.1415927, "Hello", True]
>>> print(myList[0])
>>> print(myList[3])
>>> for item in myList:
        print(item)

Record what you have learned in your word processed document.


A record is a set of related data, identified by a key value. Ideally, we should use a dictionary or even a database for this but in this case, we'll store a set of data using a simple list. Try creating the following record in Python by typing the following line of code and pressing the Enter key.

>>> customer = [154,"Mills","Mark","3","Acacia Avenue","Smallville","SM42BB","0121-2324222"

As you see, this is a normal list but all the data is related to one 'customer', me (although the address is made up). The first field is the Key Value and it's unique to this record. Using list indices, try retrieving ...


You would find this very hard if you hadn't first typed in the record or at least knew it's structure - this is why dictionaries or databases are much better to work with for this type of data storage.

Record what you have learned in your word processed document.


Finally, download the search-a-csv-file.zip archive, decompress it to your documents and Rubber DuckA plastic duck - programmers use the term when they are explaining code out loud - instead of speaking to human, the programmer will talk to a duck on his / her desk. The vocalisation of the problem often yields the solution. the script before running it. Explain how the script works in your word processed document and include a copy of the script in your notebooks.

Now print out your word processed document for your notes



Remember that lists can be one, two, three or more dimensional. If you can't remember much about lists / arrays, you might want to take a look back at the Stage 4 Lesson called 'Building data structures'.

Advanced data structures


As you will learn about these in more depth in later sections of the course, we will only introduce them here.




At some point, even more complex data structures like stacks and queues just aren't enough. When the quantity of data exceeds that which we can easily store in one data structure and relationships between data begin to become clear, the only way to go is "The Database". A database is a ...




The data stores in a database are called tables or relations. The concepts that they model are called entities. The relationship between these entities is represented by a data model or Entity Relationship Diagram.




MOVE THIS SECTION TO THE DATABASES SECTION. ADD TASKS ON CSV FILES / MASTER TRANSACTION FILES / SERIAL, SEQUENTIAL, RANDOM ACCESS FILES

Data can be stored permanently in files. Files can be...


As we are working with Python, there are two types of files we can use - text files and binary (pickle) files. You will be given examples both, including opening, reading, writing, appending, updating and closing the files in the form of scripts.

Before you start this section, create a blank word processed document with a suitable header and footer. Use screenshots and written explanation to document what you have done and what you have found out.



Open up the Python programming environment, IDLE.


Download the text-files.zip archive and decompress it into your userspace. Rubber DuckA plastic duck - programmers use the term when they are explaining code out loud - instead of speaking to human, the programmer will talk to a duck on his / her desk. The vocalisation of the problem often yields the solution. the scripts before running them. Explain how each script works in your word processed document and include a copy of the script for your notes.


Download the Binary files.zip archive and decompress it into your userspace. Rubber Duck the scripts before running them. Explain how each script works in your word processed document and include a copy of the script for your notes.

Now print out your word processed document for your notes

Estimating file sizes

File size is clearly related to the amount of data which the file contains, obvs. Often, the actual filesize is different from the value you calculate...





Data security can mean two things ...


Keeping data private from unauthorised access

Keeping your data safe must start with physical methods. When talking of hacking and data theft, it's often easy to forget that it's easier to put a padlock on the server room door than it is to implement some high falutin proxy firewall system.



Keeping data safe from accidental or deliberate loss

Backup and archiving are similar, related but different. Look carefully at the following infographic about 'Backup'.





How about these?

END OF TOPIC ASSESSMENT