How to do good research
by Gregor v. Bochmann, School of Information Technology and Engineering (SITE), University of Ottawa
(This text was prepared in October 2009 at the Hunan University of Science and Technology in Xiangtan, China)
Criteria for funding research in Canada: the most important source of funding for university-based research in computer science is the Natural Sciences and Engineering Research Council (NSERC)
- Discovery Program (unconstrained basic and applied research) – relatively small amounts of money
- Good researcher (past performance)
- Good research proposal (innovative, relevant to current state of the art in theory or practice, well explained and justified)
- Good opportunity for training of researchers at the Master, PhD and post-doctoral levels
- Collaborative research (with industry involvement) – relatively large amounts of money
- Additional criteria: potential application in the industrial context
- Evidence of industrial interest: (1) letters of support explaining relevance; (2) industrial funding: for many programs, the budget is based on matching industry funds
How to do good research : Important points (overview)
- Choose an interesting area for research
- Identify an interesting research topic (a problem for which there is no good solution)
- Have some good idea how to improve the state of the art
- Show that your idea works: e.g.
- Apply it to some examples (realistic case studies, if possible)
- Prove some properties of your approach (logical properties or analytical performance predictions) and show that it is better than the current state of the art
- Do simulation studies (e.g. for performance) and show that your approach is better than the current state of the art
- Build a software tool that supports your approach
- Do a systematic comparison with other approaches to the same problem and discuss advantages AND disadvantages of your approach
- Write up your results in some papers which make these results accessible to the interested expert.
How to do good research : more details
- Choose an interesting area for research
- Relevance for practical applications
- Area that has not yet been explored thoroughly
- Area that corresponds to your past experience (unless you want to change fields)
- Identify an interesting research topic (a problem for which there is no good solution)
- You must be familiar in general with your research area. Depending on your past experience, this may require much reading. Read surveys and overview articles.
- In order to identify your research topic, you have to look around (within the research area) to find a problem that has not yet been solved, or for which the existing solutions could be improved. For this purpose, you have to read more detailed papers. When you think, you have found a good research topic, you have to study all literature related to the problem at hand. This requires much readings. Look in good journals (e.g. ACM or IEEE Transactions) and good conferences (ACM and IFIP conferences are often better than IEEE conferences – check the acceptance rate of conferences – specialized conferences usually have more interesting papers than general-purpose conferences).
- Google is a very interesting search tool, in particular, if you know the title of the paper you are looking for. I think it is better not to rely simply on Google to do your literature search. Better: identify the relevant GOOD journals and conferences in the area and look through the published articles – read the abstract of those that have an interesting title in relation to your interest – read the whole article superficially if the abstract is promising - read the article again (thoroughly) if you are interested in the details.
- Talk to other people knowledgeable in your research area about your readings and your questions; contact the author of a paper to get a copy, or if you have questions after reading the paper in detail.
- You may find that the problem that you had chosen is already mostly solved. Maybe it would be better for you to find another topic. Go on reading.
- In the process of doing this, you may find that there is no good survey paper on the area yet. You may write such a paper.
- Have some good idea how to improve the state of the art
- Such a “good idea” comes mostly during the reading of some interesting paper, or by noting that for a given situation described in one paper, an approach presented in another paper may be useful. Most of the time, these “good ideas” do not lead to big results, but only to small improvements. Sometimes, during working on such small improvements, some further “good idea” may appear which may lead to more important “improvements”. Then you have to show that your idea “works” (see below).
- There is no recipee that always leads to a "good idea". You have to be inventive. It also helps to have a critical attitude towards the paper you are reading. Maybe it is not as simple as they say ??
- Show that your idea works: e.g.
- Apply it to some examples (realistic case studies, if possible)
- In order to check whether a “new idea” works, you should always try it out with some small examples first, and then some more complex one – if possible an example that covers all aspects of the problem.
- To convince yourself and others that your approach is interesting in practice, it is very useful to apply it to some realistic case study (this may be a prototype implementation or an extensive simulation study)
- Prove some properties of your approach (logical properties or analytical performance predictions) and show that it is better than the current state of the art
- Do simulation studies (e.g. for performance) and show that your approach is better than the current state of the art
- Build a software tool that supports your approach
- Which of the above three points is more relevant for showing that your idea works depends on the nature of your problem. Often all of these points may be pursued in parallel.
- Do a systematic comparison with other approaches to the same problem and discuss advantages AND disadvantages of your approach
- The research work under point (4) should be pursued such that at the end a systematic comparison with all other known approaches can be established. Again, a prerequisite is extensive reading in order to be familiar with the existing literature on the topic.
- Write up your results in some papers which make these results accessible to the interested expert.
- How to write a good research paper is addressed below.
- You may ask the question: Is the number of papers published important for your career ? - In Canada, it is generally considered that it is better to have a few papers in respected journals and conferences than to have many papers in journals and conferences of lower quality. There is no serious research organization that simply counts the number of papers published.
How to write a good research paper
There are a number of interesting articles on the Internet about this topic. I made copies of those articles I found most interesting (among those that I found with Google).