How to use the internet for market research

The internet offers the largest, most accessible and quickest source of data ever known. There is information out there on almost every conceivable subject, a lot of which is free. The question is how and where to find the precise information you want without drifting or drowning...

What are web directories?

Directories tend to focus on particular topics. They comprise a series of classified headings that you drill through to reach a selected list of relevant websites.

How do they work?

Directories rely on website owners submitting details of their site to them. The directory editors then visit the site and decide whether it is worthy of inclusion in the directory. Thus you are assured of a certain quality and relevance. However, the lengthy process means that directories are often not up-to-date.

Search for the right stuff

Powerful search tools make it easy, in theory, to find want you want. In practice, many searches can produce vast quantities of material, often irrelevant or unrelated to your topic of interest. Search engines keep an index of keywords on the internet. Unfortunately, as the process they use to find keywords is automatic, there is no sensible human element to filter out any garbage. Also they do not index many password-protected sites.

What are keywords?

You connect to a search tool by typing its address into your browser. You then type keywords in a search box and the search engine then sifts through its database and presents you with a list of websites that it thinks are relevant. This relevance ranking is based on the number and frequency of your keywords on the target page and where they appear. Pages with your keywords in the first paragraph are more relevant than those where the keywords appear later.

What else can search engines do?

Some search engines can answer natural language queries if you type in normal questions. Although they are easy to use and give a broad overview, you may need a more sophisticated approach to narrow your search. Easier still, most popular portals and browsers have search buttons that automatically bring up the major search tools. But you can get different results if you re-run a search five minutes later and, if it is busy, it may present you only with what it has found in the allotted time.

Try a cross engine search or umbrella site

Different search engines have particular strengths, so try several. You can automate this if you go to a 'cross engine site'. These search several search engines simultaneously, remove duplicates and present a single list of results. Unfortunately you cannot so easily fine-tune your search. Umbrella sites are where an organisation has evaluated key sources on a subject and provided links to them.

Choose your keywords carefully

The order of your keywords can make a difference so put the most important concepts first. Keep it simple: put in only the main concepts. Lock words together with quotation marks: Tony Blair or small business, for example.

Advanced search tips

Search tools have tips pages explaining other advanced techniques. These allow you to find words near each other, text in the page title, or lists of who is linked to a site. Use + to find sites containing multiple words: sailing holiday returns pages containing sailing or holiday. +sailing +holiday pulls up pages with sailing and holiday. Exclude pages with a minus sign, for example : -ocean racing will throw up pages which refer to racing but not ocean.

Access publishers' archives

Many publishers, especially newspapers, have searchable archives of past editions. These can provide very useful background information.

Ask the world

Don't forget the most powerful resource of all - other surfers. Find relevant user groups and post your query. Somebody is almost bound to know the answer or where to find it. Remember, anyone can put anything up on the web. There is no guarantee of accuracy, so try to cross-check data.

And finally, happy surfing...