Category Archives: Education & learning

Another EMIS evaluation

Next week I am off to Tonga for a couple of days. I have been asked to undertake a quick (well, two days is pretty quick!) evaluation of the education management information system (EMIS) used by the Tonga Department of Education.

This project is part of a series of EMIS evaluations in the Pacific in the next couple of months. I am off to Papua New Guinea at the end of the month.

For part of the project I will be working with Oscar, the consultant I worked with in Kiribati last year. It will be great to work with him again and on a very similar project.

I quite enjoy these projects that get me out into the real world on projects that can make a real difference to policy development and education. I also find these projects professionally rewarding in being able to use a part of my skill set that sees only limited use  in my AusAID library and knowledge services role.

Advertisements

On podcasts, learning, and uni students

When I first went to university I wrote all my essays on a typewriter.  The desktop PC revolution and word processing programs were only just beginning.  In fact, the typewriter held fast in universities throughout much of the 1980s; some universities having specific rooms full of typewriters for students to tap away upon. I had my own typewriter; but even that posed certain problems.  One memorable comment from a lecturer on one of my essays was: “I think a new typewriter ribbon would have been helpful”! 

The change from typewriter to PC and word processing was a massive change for students – a change for the best.

Nowadays, university students have it pretty easy when it comes to doing their university work.  Students have laptops that can go anywhere and word processors that make writing and editing assignments relatively painless.  Thankfully, computers made “white ink” corrections redundant.  Students don’t have to queue for ages at photocopier machines to copy key journal articles anymore – almost all academic journals these days are available online via the university library.

University students today don’t even need to go to classes to listen to lectures since many lectures are now podcast, enabling students to listen to the material at a time most convenient for them.  Moreover, podcasts overcome the multitasking dilemma we “older” students had when trying to listen to lecturers and write down salient points at the same time! Podcasting, like laptop computers, give students greater freedom and flexibility for learning.

Therefore it comes as no surprise that university students are spending less time on campus and more time online, according to this article in the Sydney Morning Herald. “Three out of four students use podcasts of lectures and a third believe online lecture materials can be a replacement for attending classes, according to the nationwide survey of 2422 first-year students by the Centre for the Study of Higher Education at the University of Melbourne”.

While we can say that technology has created the capacity for increasing a student’s online experiences, the other reason is financial.  The cost of education is much greater these days then when I first went to university (in the days when university education was nominally “free”).  Over the past 10-15 years, university students have had to pay expensive fees or take a loan to pay for their studies. For undergraduate students in particular, paying for education means there is a greater need to take part-time jobs to earn money to pay for fees, as well as the usual costs associated with text books and transport.

The negative side to all of this is the lack of on-campus activity that comes with university life.  If students are learning by themselves via online services, podcasts, and even wikis, where is the social interaction that is also part of the educational experience? 

People in the workforce do not work in social vacuums.  The lecture, with all its ancient history behind it, acted as a focal point for students to meet before and afterwards.  The death of the lecture means new focal points will need to overcome the loss of social interaction.  And I don’t just mean the fun part of social interaction; I mean actually meeting with other students (both by design and serendipity) to discuss what they have learned and what needs to be thought through.

It is interesting to compare the university experiences of students today with what will be their work experiences.  For the most part, employers want staff to be at work in a specific physical location.  Despite all the hype, working from home is still relatively rare.  Working with people at the workplace is still the fundamental organisational architecture that university students today will move into.

In the knowledge management industry, we are very supportive of face-to-face contact to establish trust to enhance workplace (working) relationships.  Trust is integral to many of the knowledge management initiatives that we like to promote, especially in terms of leadership, communication, collaboration, people networks, and operational effectiveness.

Information technology gives us speed and scale that cannot easily be replicated in face-to-face environments.  For example, knowledge management professionals like to promote online communities of practice, wikis, and podcasts too.  We see that there are many different ways to communicate information and knowledge and we try not to rely on only one channel.  We like to consider context and how our KM work best fits within that context.  In many ways, we are like both the university students and university lecturers of today’s world.

Getting the balance right in the context in which we operate is the real challenge.

On conferences

Conferences are events that I generally support because of the learning and conversations that take place.  I consider conferences to be an integral part of knowledge management, especially the person-to-person interactions that occur between sessions and at meal breaks.  The networking opportunities are also important.

But I am wondering if conferences are really all they are cracked up to be.  I heard on the news last night that there is going to be a big conference  in Canada in the coming weeks to discuss donor response to the disaster emergency in Haiti.  And in the world of international development there are always plenty of conferences taking place around the world.  Are conferences the right forum to discuss disaster relief and emergency aid when people still don’t have access to aid, food and shelter in Haiti even now?

My questioning about conferences has triggered some thoughts about networks.  The world wide web is a network of computers. An organisation is a network of functions performed by individuals, some of whom will form personal networks in order to do their jobs, and become more effective in their work.  So why aren’t networks sufficient to act in times of crises, or at other times for that matter, instead of formal conferences?  Conferences may act as a catalyst for the creation of networks, but at what point should networks replace conferences?

Whilst I have given examples from international development, the questions are just as valid for other subjects and issues.  I’d like to hear what people think about this conference issue, and whether there is any scope for networks to take over.

On flickr and development agencies

There’s an interesting blog post from Timo at the Red Cross about the use of Flickr to showcase the international development and humanitarian work done by that agency (thanks Nadejda on KM4dev for the tip).  The Red Cross Flickr stream is really a terrific site and well worth a visit.  Where I work, AusAID has a Flickr site too.

Timo’s blog post cites eight lessons learned from the experience of using Flickr:

1. know your audience
2. newsworthiness beats quality
3. less is more
4. understand what you want to achieve
5. use Flickr groups
6. appreciate the work of others
7. need to give solid attention to Flickr to maintain traffic
8. be careful with creative commons licensing

What is missing, and Timo alludes to this in his blog post, is that Flickr needs better integration with other applications. Timo suggests that Flickr needs to better integrate with Facebook, for example.  In addition, I think we also need to work out how better to use Flickr to tell the stories behind the photos.  I still feel that the images, words and tags are not enough to really give me a strong sense of place and story.  There is greater potential for education and learning beyond just the images themselves, albeit I know how powerful images can be in their own right.

It would be great to be able to link the photos to a short podcast, perhaps a narrative fragment from one of the image subjects, to really give stronger context to the individual images.  Not sure if this is possible, but I am certain narrative would add to the user-experience.

On outcomes and impact

There are many ways to find out about things. Research is obviously part of that. And research likes to use quantitative measures in order to maximise objectivity, even if these measures don’t give you much meaning.

Let’s look at hit rates on a website – a metric commonly used for “statistical purposes”. What does it mean? Well, it means that a website or page view has been looked at a certain amount of times. The inference is that the more hits you have the better must be the result. But what is the result?

If the intended result is to have as many hits as possible since one assumes hit rates equate with “eyeballs”, then surely high hit rate numbers are great. But is this the result an organisation really wants from it’s website? What happens as a consequence of the “eyeballs” is the question I really want to get an answer to. In reality, high hit rates could indicate a bad website. Your website visitors and customers are clicking away, frustrated by their inability to reach an outcome they want to achieve. Just get those click rates up and everything will be fine….hmmmmm.

Let’s do a survey then. A survey is actually pretty limiting.  A questionnaire is bounded by the construction of the questions and limited answer options. In nearly all cases, one could answer a question yes or no, depending on the particular circumstances at some point in time. Surveys also don’t do a great job in measuring continuous change over time. And conducting surveys or focus groups with large numbers of people are often difficult and time consuming, certinly if a continuous process is required.

Yet these methods are still held to be superior to more qualitative approaches to research. However, if you actually asked your website users what they thought of the website, perhaps they might tell you that it takes a lot of clicking to get through to complete the task at hand. They might tell you that your website is poorly organised, with lousy navigation and confusing labels. They might tell you that the photos on the home page add nothing to their customer experience. They might tell you that your website could better… for them. And if you have a continuous dialogue with them, they will be even more insightful as to how to improve or validate what you are trying to do. Observation at point of impact is a good way of thinking about this.

I can see some meaning from getting those kind of responses! Click rate numbers – forget it. Now I have real information that can make an impact to the people I say (and the organisation) I am trying to serve.

So what we are interested in finding out is impact. What is the  impact that occurs from what we are doing? This is different to outcome. Click rates are an outcome. Obtaining continuous feedback to ensure satisfied customers buy from you, recommend you, and stay loyal, is another.

Now what if we could get this feedback quickly, continually over time, on a large scale, context-sensitive, and in a way where the person giving us the information gives it in terms of how it affects them, and not through some intermediary or stilted survey method?

I set the scene this way to introduce some thoughts from a presentation at the ANU yesterday by Dave Snowden,  special guest speaker at the ACT-KM forum. Dave talked about a number of current projects he was working on. The common element from his talk was the importance of  determining impact and how then to take relevant action as a consequence.

I will use the example of the Liverpool Slavery Museum in the UK from Dave’s talk yesterday; albeit the Children of the world project was for me the most fascinating.

One could count the number of people going through the museum each month and year. The numbers might indicate level of popularity but one can’t be sure. At best, they show that “x” number of people came and paid “y” number of UK pounds to do so.  One could do a simple accounting calculation at this point and perhaps leave it at that.

But what if you wanted to know what effect the museum had on people? What if you wanted to know how successful the museum was in educating visitors about slavery, or in providing a unique experience? What really was the impact of the museum visit?

[It is of course true that if you don’t want to know about your customers’ experiences and are happy with just throughput figures – akin to an assembly line – then impact will have very little interest for you. The process will be sufficient].

Dave told us how there are computer screens and keyboards at the museum where people can record their experiences and feelings about the museum exhibits. People can nominate any of the individual exhibits to make a comment or express a feeling. The people making these comments are then able to “index” or tag their comments using terms chosen freely that signify meaning to them. Nobody  is interpretating what they say and adding any bias. At the same time, this information capture is continuous and provides for scale, something a static survey couldn’t do. The museum now has thousands of narrative fragments “indexed” by the individuals themselves. This information is aggregated and patterns observed. These patterns may suggest a change to a particular exhibit, or perhaps some alteration to how a museum education officer conducts a group tour.

In the Liverpool Museum case, they have both quantitative information (number of visitors and monies received) AND what impact the museum had on the visitors.

Yet still there are detractors:

  • stories (narrative fragments is the preferred method used by Dave Snowden) are not real facts
  • some people may just write junk and not tell the truth
  • it’s all so subjective

All three statements might be true. The point is, if we want to measure impact, then we need to know what people think and what effect something had on them. And we need to know what they think, not what what we might guess at. The capture and aggregation of narrative fragments is a good way of doing this. “Junk” can be easily discarded but sometimes “junk” may be of interest as a weak signal, something we should pay attention to. Where you want to establish an impact on people, of course there is subjectivity. However, how the narrative fragments are captured, aggregated and used is quite a rigorous and objective method in itself.

Lastly, no matter what the method, unless people use the tools correctly and respond appropriately, no research activity will have any validity.

On knowledge management and human resources

Some of you may already know that I am a great believer in the interplay between knowledge management (KM) and human resource management (HRM). I consider both KM and HRM as catalysts and facilitators for developing and enhancing an organisation’s intellectual and social capital. I believe KM is more than just managing databases and HRM is more than just payroll.

KM and HRM actually have a number of responsibilities in common: learning and development, enhancing staff performance, staff retention, team building and effectiveness, organisational development, and cultural change. The approaches may differ but I believe that the strategic and practical outcomes are the same – to equip individuals, groups, and the organisation to innovate, solve problems, establish effective workflow, and deliver quality services and outputs (both internally and externally).

I was therefore very pleased when I found out about a post-graduate course being run from Lancaster University Management School in England. The course is a Master (MA) in Human Resource & Knowledge Management. The course looks at “the conditions in which HRM and knowledge management have become the central links between people, work, and technology in contemporary organisations”. Excellent!

I wonder if there are any other examples of KM and HRM being featured within the same training or academic course.

Virtual Participation Camp: Changing the Rules, June 27-28

Having just blogged about conference formats, I have just received the following email from Stephen Dohrn via the km4-dev list-serv. I hope Stephen doesn’t mind me reproducing most of that post here. The virtual participation camp sounds like it could be a very different conference experience.

Participation Camp, Change the Rules, in New York on June 27-28, will provide the spark for an explosion of sharing, experimentation and collaboration. Democracy is the game where we can change the rules together! How do we make this game more serious, more fair and more fun? Please let us know if you are interested in convening a virtual session at this event on a topic of your choice, or collaborating with us in some way!

Participants may attend a wide range of physical and virtual presentations (or deliver one themselves), compete in a conference wide web participation game called Nomic (http://en.wikipedia.org/wiki/Nomic), or roll up their sleeves in a hands on workshop. For preliminary details see: http://participationcamp.org/.

If you might be interested in collaborating with us, please check out the wiki at: http://barcamp.pbworks.com/ParticipationCamp

What Makes Change The Rules Different?

Virtual/Physical Hybrid Structure: One particular feature of this event is that we will be bridging the physical and virtual worlds. We will be opening up virtual spaces in advance of the actual session so as to engage virtual participants in the project. We will also have a room where virtual presenters can connect with those at the conference.

Open Space/Defined Hybrid Structure: We will be using Open Space principles for the creation of some of the sessions, but will also be seeking out the involvement of those that would like to actively
engage participants on a specific topic. If there is an issue or a question that you would like to discuss at this event, please let us know!

Play Game: We will be playing the game, Nomic (http://en.wikipedia.org/wiki/Nomic). This is a game that is designed to teach participants, by virtue of their experience, some of the interesting features of governance, democracy, rule making, rule following, collaboration etc.

Pre-Session Dialogue: We will open SkypeChat spaces that enable those that are interested in the PCamp theme(s) to connect with each other, exchange ideas, plan potential sessions etc.

Sustain Dialogue: Due to the fact that virtual environments are accessible from anywhere, it becomes possible for participants to continue their conversations with others after the conclusion of the
session. This makes it possible for them to continue to explore the ideas and projects that they are interested in, as well as to cultivate the relationships with those that they have connected with.
We hate the fact that what happens when events end is that there is little or no follow up!

Questions/Themes

Here are a few questions that we have been thinking about. Are there any such questions that are of interest to you, and around which you might be interested in organizing a virtual session?

What, generally speaking, is the role that technology can play in fostering citizen engagement?
What are the best tools for creating the right frameworks for fostering citizen engagement?
What are the particular challenges of using open, collaborative, platforms?
What sorts of business models are consistent with ‘open collaboration’? How can organizations that subscribe to these principles also generate revenue?
How do we utilize technology to mobilize the youth vote?
Process

On June 20th, we will create a chat space/conversation in Skype to which we will invite all those that are interested in participating in an open dialogue on issues relating to open governance. This chat
space will allow you the opportunity to:

Introduce yourself and your project to others that are like minded
Connect with others that might be interested in your project or might have interesting project ideas.
Learn, via participation, about how open, collaborative, patterns of interaction work
Learn, via participation, how groups self organize
Virtual Tools

We will be using free online tools that are easily accessible by any participant, such as:
Drop.io: We will utilize drop.io (www.drop.io) in order to organize and share files.
Google Documents: will be used for the joint authoring of documents.
SkypeChat: Will provide a open space where people can start the dialogue, network and keep the discussion going.
Etherpad: for notetaking during sessions.
Twitter: as a channel to the outside to integrate other interested parties.
—————————————————————————————————–

You can’t get more web 2.0 and participatory than this conference!