Competitors price monitoring on the Internet - methods and programs

How does the price affect the buyer's decision?

In our publications, we have repeatedly written about the importance of analyzing prices in the market and the general market situation.
In recent years, in open sources, there is a lot of information about the influence of price on the choice of a potential customer to buy goods in a particular store. All of them emphasize the importance of understanding competitors' pricing on sales and profits of each seller online. Monitoring the prices of competitors in this case becomes the main weapon of competition. Here are some facts.

A survey was conducted in the US a few years ago. It was conducted among people making their purchases through their smartphones. The first thing that caught the eye of the researchers was that 85% of buyers under the age of 35 compared the offer prices on various online sites before buying. Many of them even resorted to using special online price comparison services that monitor prices in online stores for the product chosen by their client. The main result of the study is as follows - 81% of potential buyers chose the site with the lowest price for their chosen product.

ВAll of the above facts suggest that the most important factor influencing the buyer's choice of the seller is the price. To establish and maintain a competitive price, you need to know the prices of competitors. There is nothing to do, we start monitoring the prices of competitors. 

Supplier price monitoring

Supplier price monitoring is a process of regular systematic collection and subsequent study of all price offers of online store suppliers: special and promotional offers, discounts and bonus programs to find the minimum purchase price for similar products.

Each online store has dozens, hundreds, and sometimes thousands of suppliers of goods for sale. It is not uncommon for a store to purchase the same product from several distributors. They compete with each other, trying to offer stores the most favorable conditions for the purchase.

It is important for an online store to constantly monitor prices and track the availability of goods from each supplier in order to determine where to buy goods today is most profitable. It is important for the buyer of an online store to see the full picture of the suppliers' offer in a form convenient for analysis and decision making. It is quite difficult, and often impossible, to solve this problem manually or with the help of price parsers. To solve it, it is more profitable to use the products of services that offer services for monitoring the prices of competitors and suppliers. The Price Control project has good experience in this matter.

How to monitor competitor prices

The competitor price monitoring program contains several sequential steps to determine the competitor segment, connect the price monitoring process and develop marketing pricing solutions.

So how do you monitor competitor prices?

There are several methods to find out the prices of competitors:

• Manual price monitoring.
• Telephone monitoring of retail prices
• Market monitoring in accounting systems
• Automatic price monitoring (parsing)

I propose to consider the fundamental methods of monitoring on the Internet. I will describe these methods in order of increasing complexity and cost.

Manual price monitoring

ПIt suitables for small online stores with a close circle of competitors and a small range of goods or services. Company employees manually, through their computers or mobile phones, go to competitors' websites and collect their prices in Excel spreadsheets to perform the task of monitoring prices, Then they pass the results of the monitoring to the management to make a decision on changing the company's own prices. This process can also be called price monitoring in Excel.
The downside of price monitoring in Excel is that it takes a lot of time and effort. It requires daily visits and monitoring of price changes in competing stores. This work is assigned as an additional burden to the main activity and assigned to random employees. Usually they are not enthusiastic about this activity, which affects the overall quality of the price monitoring results. If you decide to expand the range of products, then the cost of monitoring resources will also increase, which will require the hiring of additional employees.
Price monitoring of competitors can also be carried out using price aggregators (Hotline, Google Shopping, etc.). Using it is quite simple: you enter the name of the product in the search bar, and the system displays all available offers.
The disadvantage of this method is that these sites are 80% advertising. Not all of your competitors present their products there. Their prices do not always correspond to the real sale prices due to their "advertising" function and by clicking on the link to the site, you may see a completely different offer.

Many online stores deliberately underestimate the prices of their offer - heavily dumping on price aggregators. Their prices try to be the cheapest there in order to collect the maximum traffic of potential buyers. Some sellers advertise on price aggregators for products that they don't even have in stock. When contacting potential buyers who have clicked on this advertisement, they try to switch them to the purchase of analogue products.

Telephone monitoring of retail prices

This method of obtaining reliable information is rightfully considered one of the most effective, but requiring the most time and labor. Managers of a competitor company will be happy to share data on prices, special offers, upcoming promotions and a loyalty program with "potential customers".
The disadvantages of this method are that it requires the acting skill". You need to be able to express sincere interest and pretend that you are “floating” in this topic so as not to arouse suspicion. This method is well known to your competitors, so you need thorough preparation: think about what questions you will ask, what contact details you will leave (phone number and e-mail that are not related to your company, where your object is located, when you need to make a delivery, need delivery or pick it up yourself ...) and follow the speech without using professional terminology.
To carry out telephone price monitoring, trained and changing employees are needed. Competitor sellers will be “surprised” by weekly calls from the same “potential buyer” who constantly asks for information about prices, discounts, promotions, bonuses, and does not buy anything. The process of telephone monitoring itself takes a lot of time for your employees, but allows you to build a pricing strategy based on data on the real prices of competitors.

A bonus to telephone monitoring may be the ability of our own staff to carry out a control purchase of goods from competitors. This option is useful for finding out the channels for supplying products to your competitors' outlets.

You can read more about the possibilities, methods, methods and results of a test purchase in our article "What is a test purchase in an online store and what tasks it helps to solve."

Price monitoring of marketplaces and price aggregators

Competitor prices can also be monitored using price aggregators (Hotline, Google Shopping, etc.) and marketplaces (Rozetka, Prom.ua, etc.). Using it is quite simple: you enter the name of the product in the search bar, and the system displays all the available offers.

The disadvantage of this method is that these sites are 80% advertising. Not all of your competitors present their products there. Due to their "advertising" function, their prices do not always correspond to the real sale prices and by clicking on the link to the site, you may see a completely different offer.

Many online stores deliberately lower the prices of their offer - heavily dumping on price aggregators. They try to be the cheapest there in order to collect the maximum traffic of potential buyers. Some sellers advertise on price aggregators for products that they don't even have in stock. When contacting potential buyers who have clicked on this advertisement, they try to switch them to the purchase of analogue products.

You can learn more about how price data is processed and how price analysis of competitors works with the Price Control service from our article "Analysis of competitors' prices on the Internet market".

Price monitoring Rozetka.
What to look for in the marketplace?

When monitoring the prices of sellers on the Rozetka marketplace, it is necessary to carefully analyze the collected data. Namely, pay attention to the availability of goods. It has long been no secret that merchants have limited access to the functionality of removing products from the site or changing prices for it if it is not available for sale. There are times when a merchant does not have a single product in stock, but Google search results show it for a relevant query.
If the price monitoring program you are using is not configured to collect availability data, then the results obtained may be considered outdated. If you receive a monitoring report without information about the availability of a product, you will be very surprised how many of your competitors on the marketplace sell the same product as yours at prices below the purchase price.
Without taking into account the factor of availability of goods, the analysis of competitors' prices will be erroneous. Therefore, it is recommended to carefully choose a program or service provider for monitoring prices on the Internet. Analyze your tasks and make sure that the service you have chosen can solve them.

The methods described above involve significant time and financial costs. It is more efficient to use the second group of methods for this purpose - automated systems for monitoring competitors' prices.

Automatic monitoring of competitor prices

Automatic monitoring of competitors' prices is a regular process of collecting prices and stocks of goods on various Internet sites, where they change very often, without the participation of company personnel. Without automated price monitoring systems, it is difficult to track and respond in a timely manner to the rapid change in the market situation in highly competitive industries.

Market monitoring in accounting systems

One of the best solutions in monitoring issues can be the installation of additional modules for your accounting system, for example, 1C. This is not one module, but a whole complex that includes all kinds of price research methods from manual processing of sites to parsing competitors.
This method allows you to quickly take action and adjust your own prices. For the convenience of analytics, reports are generated in XML or JSON format.
There are also disadvantages in this case. Item cards are created differently, so the names of the same products may differ. This makes it particularly difficult to monitor commodity prices. Therefore, even here one cannot do without “manual labor”: find the appropriate position and only then monitor price changes.

A bonus to telephone monitoring may be the ability of our own staff to carry out a control purchase of goods from competitors. This option is useful for finding out the distribution channels of products to your competitors' outlets.

Monitoring prices on competitors' websites using a parser

The most effective way to monitor prices is automated parsing using special software - a parser.
The competitor's price parser is a specially designed computer program designed to collect data on the cost of goods or services from the pages of the sites visited by the parser. This program is able to bypass crawling protection, and has fine settings to collect exactly the information and in the form specified by the customer.
Parsing competitors' prices is an automatic process of monitoring the prices of competitors or partners, which is performed regularly, according to a schedule. The purpose of price parsing is to obtain up-to-date information to control prices on the market and help the customer optimize their own pricing policy.
Competitive price monitoring software for online reseller allows you to compare prices of online stores selling the same products as your company and work out the best price for your offer. The price parser of competitors for suppliers will allow you to compare the offer of competitors to analogues, and understand how your price is more (or less) attractive to customers.

Automating price monitoring using a parser has several solutions.

1. Buying a ready-made parser for price monitoring.
2. Development of own parser.
3. Cloud services for monitoring competitors' prices.

Let's briefly consider the possibilities of each of these methods.

Buying a ready-made parser based on tables.

If the issue of monitoring competitors' prices has become acute and there is no time to develop your own, you can buy a ready-made parser and customize it for your own tasks.
The monitoring results are collected by these parsers in Excel tables and Google sheets.
The action of these parsers is based on macros of automated actions or special formulas that allow you to collect data from sites.
This type of parser is suitable for solving simple tasks for collecting data on resources that are not subject to frequent changes. Macros need to be customized. Output results in tabular format only.

Development of own parser.

Writing a parser to solve your own problems is more preferable than buying a ready-made one. But there are also nuances here. The first is who to involve in writing a parser for monitoring prices: a freelancer or a company that provides such services? The second nuance is who will maintain and regularly reconfigure the competitor's price parser in case of changes in the layout of online store sites or protection against parsing?
(For more on scraping issues, see 9 Barriers to Consider When Scraping Competitor Prices from Websites.)

Payment for the services of customizable competitor price monitoring services

Price monitoring services can be ordered from specialized services. These customizable services typically offer turnkey solutions. A clear advantage of such competitor price monitoring services is the specially developed software for collecting data from even the most complex and secure sites.

In addition, such services regularly track changes in the settings of sites that are included in the monitoring and reconfigure the parser to collect up-to-date information. The cost of services of such services is much lower compared to the costs, for example, of maintaining your own programmer, servers, etc. or paying manual labor to employees.

We provide descriptions, characteristics, application features and pricing policy of the TOP 5 similar platforms in the material about the price monitoring services of competitors on our blog.

For example, the Price Control service offers not only price monitoring services, but also the development of an individual solution for the client's business tasks. The service "Monitoring prices of Marketplaces" has been especially popular lately. This is due to the rapid development of Marketplaces - as a priority for many sales channels on the Internet.

Online price monitoring of competitors - is it legal?

In this section, I want to say a few words about the legality of data parsing and brand protection on the Internet. Lawyers argue that it is worth paying attention to the legislation on information. As a rule, the legislation of many countries defines information as an object of public, civil and other legal relations. Information can be freely used by any person and transmitted by one person and or other requirements for the procedure for its provision or distribution. Based on the foregoing, information, depending on the category of access to it, is divided into public information, as well as information, access to which is limited by federal laws (restricted access information). Further, the law states that information, depending on the procedure for its provision or distribution, is divided into: information freely distributed, information provided by agreement of persons participating in the relevant relations, information that is subject to provision or distribution, information, the distribution of which is limited or prohibited.
In most countries, there is no law restricting access to price information in stores. Thus, this information is publicly available, which means that it is not prohibited to collect prices in the online store.
The law allows everything that is not prohibited. Automatic price monitoring will be legal if it does not violate the prohibitions established by law. This means that when parsing information from Internet sites, it is necessary to comply with current legislation.
There are a number of restrictions regarding the parsing of information from sites and the protection of intellectual property on the Internet: you cannot violate copyright and related rights, you cannot collect legally protected information, you cannot collect data constituting a trade secret.
Based on the principle, everything that is not prohibited is allowed, automatic collection of information (web scraping) is not illegal

• if information is in the public domain and is not protected by copyright and related rights.
• • Parsing is carried out in compliance with the law.
  • Price monitoring does not disrupt the site.

When automatically collecting information from sites, it is important:

1. Do not collect copyrighted content.
2. Do not disrupt the site during parsing.
3. Do not violate the terms of use of the site.
4. Do not collect personal information of users..

Sometimes there are complaints from sites that are exposed to automatic parsing. They argue that the work of parsers interferes with the operation of the site, increases the load on the server, and leads to additional costs for maintaining the site's performance.
This is done both by Google and Yandex, which are engaged in regular parsing (indexing) of the entire site, and by the mass of competitors who analyze other people's content, by brand reputation monitors, and by potential and real customers of the site ... Prove that it is the parser, collecting price from the site, interferes with its work will be difficult. Services that professionally provide price monitoring services are able to configure their robots so that they have a minimal impact on the operation of the site and do not fall under its protective sanctions.