Monitoring competitors' prices on the Internet - methods and programs

price monitoring

Price monitoring is a regular analysis of the price and assortment policy of retail outlets and retail chains in order to track market trends in order to build your own price and marketing policy. The company’s pricing policy is one of the determining factors for a successful business.
Price monitoring on Internet is the process of searching and collecting prices for goods in online stores, price aggregators and marketplaces. Competitive price monitoring is necessary for every online business in order to form the optimal own price, and partner price monitoring will allow manufacturers and suppliers to monitor compliance with the MAP/ MSRP.

Successful sales in the e-commerce segment directly depend on the price on the market at that moment. Believe me, a potential buyer will not be lazy and in 2 clicks will compare prices in your store and those of competitors, and will go where the offer is most profitable. Therefore, price monitoring should become an obligatory routine, as one of the keys to the success of any online store, as well as each online supplier.
Competitor price monitoring is a regular monitoring of prices, promotions and the availability of identical or similar goods on the pages of their websites. The purpose of monitoring competitors’ prices is to regularly re-evaluate own products, based on the market situation, in order to increase profits. The main tool for automatic collection of information is the parsing of competitors’ prices.

What is monitoring

How Wikipedia defines the term “monitoring”.

Monitoring is a system of continuous monitoring of phenomena and processes taking place in the environment and society, the results of which serve to substantiate management decisions to ensure the safety of people and economic facilities. Within the framework of the monitoring system, an assessment, control of the object, management of the state of the object takes place, depending on the impact of certain factors.

Price monitoring is a systematic collection and analysis of online prices of customers, suppliers and competitors. Pricing information is used to generate your own prices and, if you are an online provider, suggested retail prices (MAP/ MSRPs) for partners. The term “price tracking” can be used instead of the term “price monitoring”.

Price monitoring is not a newly invented process. Even before the advent of online shopping, employees of retail chains went to competitors’ stores and tracked the prices of goods. The data was not even recorded in a notebook, becouse it would be suspicious, but simply remembered.
The purpose of such monitoring was a timely response to market trends. In this way, companies could to form competitive pricing strategies.
Unfortunately, the information manually collected by traders could be outdated before they began to analyze the data. Competitors by this time could either change prices or apply discounts to the product of interest.
The advent of the Internet and retail chains’ own websites has significantly accelerated the process.

  • Employees don’t have to travel to retail stores.
  • Price data can be obtained in real time.
  • It is possible to analyze the pricing strategy of competitors within a few minutes.
  • Up-to-date data on the pricing policy of competitors allows you to develop effective business decisions.
  • Now you can implement dynamic pricing.

How price monitoring helps in developing a company’s marketing strategy, I will discuss further.

Price monitoring and marketing

commodity price monitoringMarketing is the maximum satisfaction of customer needs with maximum profit for the company.

Monitoring of price is a systematic collection of offer prices of all competitors and customers of the company, monitoring changes in the market, building a forecast of the situation in the future. The company’s maximum profit from the sale of its products can only be obtained if its own pricing strategy is verified and the best MAP/ MSRPs are recommended to its partners.

Price slice is a method of market analysis, when at a given point in time, we get all the information we are interested in about prices, promotions, discounts and the availability of goods on the websites of all the trading platforms we are interested in. Regularly conducting such an analysis, you can track the trends and patterns of the market, adjust your own pricing strategy.

The goals of price monitoring in marketing are:

  • • collection of all available information about competitors’ prices;
    • forecasting changes in these prices;
    • building hypotheses about competitors’ pricing strategies;
    • developing your own pricing strategy.

Price monitoring should solve the following tasks:

  • • find the main competitors of the company;
    • systematically monitor their prices;
    • collect statistics on price changes;
    • make an analysis of these changes and the factors influencing them;
    • identify key trends;
    • give a forecast of changes in competitors’ prices for the future.

Where is it important to monitor prices on the Internet:

  • • in the online stores themselves;
    • on price aggregators;
    • on marketplaces;
    • in advertisements in search results;
    • on various electronic bulletin boards;

Book a consultation with our specialist

    call centre

    Features of price monitoring in eCommerce.

    When did price monitoring appear?

    Since ancient times, all traders have been monitoring prices. Before taking their goods to "distant countries", they asked all visitors for a long time and carefully about the prices in the markets of these countries, sent their workers there, and traveled themselves. Arriving at the bazaar, they always walked through the ranks of competitors, looked at prices and the supply of goods, conferred for a long time and only then set prices for their goods and started trading.

    trade war
    Merchants also had problems with low-quality or outdated price monitoring. In ancient times, caravans and ships traveled for a long time, couriers and messengers also did not move quickly, information about prices and competitors could become outdated. Merchants come to the fair with their goods, and there are already competitors who came earlier with similar goods. Price wars began, but they could beat them with sticks.
    Now is the time of rapid development of Internet commerce. And the sale of goods and, accordingly, the monitoring of competitors' prices are moving online. Price information is available to everyone in real time. To change prices on any electronic trading platform, it is enough to press a few keys and this happens in a few seconds. Therefore, now the timeliness of monitoring competitors' prices and the speed of reaction to their changes are of particular importance for every company selling its products online.

    Why price monitoring helps to earn?

    Price monitoring analysis is a procedure for a complete analysis of competitors' price data, manufacturers' offers, updating one's own price policy in comparison with the profit margins of internal processes and market goals. As a result of the analysis of price monitoring, a decision is made on pricing in the company.
    Automatic monitoring of competitors' prices is a regular process of collecting prices and stocks of goods on various Internet sites, where repricing can take place within a few minutes, often without the participation of company personnel. Without automated price monitoring systems, it is difficult to track and respond in a timely manner to the rapid change in the market situation in highly competitive industries.
    Below I will give examples when the exact knowledge of the prices of competitors' products helps to earn more.

    faktsExample 1.

    You sell a drill for 1-2% more than in another online store. Even a lazy customer will take two minutes to compare your offers. Ceteris paribus, the choice of the client is obvious. He will buy where it is cheaper. You incurred the cost of introducing and advertising the product, and the competitor received the income.
    Now let's assume that you are the only seller on the Internet in this region who still has this drill in stock at the moment. If you have timely monitored the prices and stocks of competitors and you know exactly this situation, you can raise the price for the duration of your exclusive position. There is a high probability that the client will be willing to pay more and you will earn more.

    faktsExample 2.

    You and your direct competitor in a given region sell products from the same supplier. You have noticed that your sales have started to drop. The results of price monitoring showed that the competitor on weekends and holidays lowers the prices for these products to your purchase price or even lower. You verified the data with a test purchase from a competitor. Error options are excluded. The conclusion is unequivocal - your competitor does not work at a loss, but has an additional discount from the supplier. You contact the supplier, negotiate. Either you will receive a discount and be able to trade at a competitor's prices, or you will change the supplier for these products, or you will take on board analogues with a guaranteed markup.

    faktsExample 3.

    You are one of the dealers of a major European supplier of garden care products. The product is difficult to operate and maintain, so most of the assortment is prohibited for sale via the Internet, and should be sold only in offline dealerships with the advice of certified specialists. The supplier has strict requirements for the assortment, stocks, organization of a sales office and a service center. Investments in business are constantly increasing, but suddenly incomes began to fall. You ordered a search for the presence of goods and monitoring their prices on the Internet. The result of the monitoring showed that on the websites of some of your fellow dealers there appeared products that are prohibited for sale online, and even at a dumping price. You filed a complaint with the supplier, provided monitoring data, screenshots of site pages with prohibited goods. The supplier fulfilled its obligations and terminated the contract with the violating dealer. For a while, there was one less competitor in the market, and you grew in revenues and profits, and possibly in market share.

    Competitor price monitoring is essential,
    • • when you are planning an advertising campaign. It will not bring success without a competitive price.
      • when prices play a major role in a potential customer's purchasing decision. Your price gap will cover most of your sales.
      • when you develop a strategy for promoting your company in the market. Better prices for your customers can create a temporary competitive advantage;
      • when you need to sell off an old collection or reduce your total stock of items. The lowest prices will help you quickly attract potential customers.


    Find out who is selling your brand right now

    Order a test service

      Order a test service
      10:00 - 19:00 (mon-fr)

      How does the price affect the buyer's decision?competitors price monitoring

      In our publications, we have repeatedly written about the importance of analyzing prices in the market and the general market situation.
      In recent years, in open sources, there is a lot of information about the influence of price on the choice of a potential customer to buy goods in a particular store. All of them emphasize the importance of understanding competitors' pricing on sales and profits of each seller online. Monitoring the prices of competitors in this case becomes the main weapon of competition. Here are some facts.

      factsA survey was conducted in the US a few years ago. It was conducted among people making their purchases through their smartphones. The first thing that caught the eye of the researchers was that 85% of buyers under the age of 35 compared the offer prices on various online sites before buying. Many of them even resorted to using special online price comparison services that monitor prices in online stores for the product chosen by their client. The main result of the study is as follows - 81% of potential buyers chose the site with the lowest price for their chosen product.

      ВAll of the above facts suggest that the most important factor influencing the buyer's choice of the seller is the price. To establish and maintain a competitive price, you need to know the prices of competitors. There is nothing to do, we start monitoring the prices of competitors. 

      Internet price monitoring program

      Internet price monitoring programSuccessful sales in the e-commerce segment directly depend on the price on the market at that moment. Believe me, a potential buyer will not be lazy and in 2 clicks will compare prices in your store and those of competitors, and will go where the offer is most profitable. Therefore, monitoring the prices of competitors should become an obligatory routine, as one of the keys to the success of an online store.

      How to track competitor prices

      Price tracking is a regular collection of information about changes in the cost of a product of interest to a potential buyer on the Internet. He can do it himself, regularly visiting sellers' websites, he can entrust tracking prices of online stores to special services. The program for comparing prices on websites will automatically track the price of the product and give the customer the best offer.

      The price analyzer can both monitor the prices of a single marketplace and collect prices from different sites. He can give a cut of prices on request, or he can send information to the customer, for example, when the price of a product drops to a certain level.
      It is also reasonable to use market monitoring for commodity producers when introducing a new product for sale. This can reflect how successful the promotion is and what shortcomings have been identified during marketing campaigns.
      To analyze the prices of competitors, it is necessary to perform several sequential steps to determine the segment of competitors, connect the price monitoring process and develop marketing solutions for pricing

      So, how do you track competitor prices?

      There are several methods to find out the prices of competitors:

      • Manual price monitoring.
      • Telephone monitoring of retail prices
      • Market monitoring in accounting systems
      • Automatic price monitoring (parsing)

      I propose to consider the fundamental methods of monitoring on the Internet. I will describe these methods in order of increasing complexity and cost.

      Manual price monitoring

      price monitoring program

      ПIt suitables for small online stores with a close circle of competitors and a small range of goods or services. Company employees manually, through their computers or mobile phones, go to competitors' websites and collect their prices in Excel spreadsheets to perform the task of monitoring prices, Then they pass the results of the monitoring to the management to make a decision on changing the company's own prices. This process can also be called price monitoring in Excel.
      The downside of price monitoring in Excel is that it takes a lot of time and effort. It requires daily visits and monitoring of price changes in competing stores. This work is assigned as an additional burden to the main activity and assigned to random employees. Usually they are not enthusiastic about this activity, which affects the overall quality of the price monitoring results. If you decide to expand the range of products, then the cost of monitoring resources will also increase, which will require the hiring of additional employees.
      Price monitoring of competitors can also be carried out using price aggregators (Hotline, Google Shopping, etc.). Using it is quite simple: you enter the name of the product in the search bar, and the system displays all available offers.
      The disadvantage of this method is that these sites are 80% advertising. Not all of your competitors present their products there. Their prices do not always correspond to the real sale prices due to their "advertising" function and by clicking on the link to the site, you may see a completely different offer.

      Many online stores deliberately underestimate the prices of their offer - heavily dumping on price aggregators. Their prices try to be the cheapest there in order to collect the maximum traffic of potential buyers. Some sellers advertise on price aggregators for products that they don't even have in stock. When contacting potential buyers who have clicked on this advertisement, they try to switch them to the purchase of analogue products.

      Telephone monitoring of retail prices

      This method of obtaining reliable information is rightfully considered one of the most effective, but requiring the most time and labor. Managers of a competitor company will be happy to share data on prices, special offers, upcoming promotions and a loyalty program with "potential customers".
      The disadvantages of this method are that it requires the acting skill". You need to be able to express sincere interest and pretend that you are “floating” in this topic so as not to arouse suspicion. This method is well known to your competitors, so you need thorough preparation: think about what questions you will ask, what contact details you will leave (phone number and e-mail that are not related to your company, where your object is located, when you need to make a delivery, need delivery or pick it up yourself ...) and follow the speech without using professional terminology.
      To carry out telephone price monitoring, trained and changing employees are needed. Competitor sellers will be “surprised” by weekly calls from the same “potential buyer” who constantly asks for information about prices, discounts, promotions, bonuses, and does not buy anything. The process of telephone monitoring itself takes a lot of time for your employees, but allows you to build a pricing strategy based on data on the real prices of competitors.

      A bonus to telephone monitoring may be the ability of our own staff to carry out a control purchase of goods from competitors. This option is useful for finding out the channels for supplying products to your competitors' outlets.

      You can read more about the possibilities, methods, methods and results of a test purchase in our article "What is a test purchase in an online store and what tasks it helps to solve."

      Price Monitoring Program

      The methods described above involve significant time and financial costs. It will be more effective for this purpose to use the second group of methods - automated price monitoring systems.

      Market monitoring in accounting systems

      ОOne of the best solutions in monitoring issues can be the installation of additional modules for your accounting system, for example, 1C. This is not one module, but a whole complex that includes all kinds of price research methods from manual processing of sites to parsing competitors.
      This method allows you to quickly take action and adjust your own prices. For the convenience of analytics, reports are generated in XML or JSON format.
      There are also disadvantages in this case. Item cards are created differently, so the names of the same products may differ. This makes it particularly difficult to monitor commodity prices. Therefore, even here one cannot do without “manual labor”: find the appropriate position and only then monitor price changes.

      A bonus to telephone monitoring may be the ability of our own staff to carry out a control purchase of goods from competitors. This option is useful for finding out the distribution channels of products to your competitors' outlets.

      Automatic price monitoring (parsing)

      automatic price monitoringThe most effective way to monitor prices is automated parsing using special software - a parser.
      The competitor's price parser is a specially designed computer program designed to collect data on the cost of goods or services from the pages of the sites visited by the parser. This program is able to bypass crawling protection, and has fine settings to collect exactly the information and in the form specified by the customer.
      Parsing competitors' prices is an automatic process of monitoring the prices of competitors or partners, which is performed regularly, according to a schedule. The purpose of price parsing is to obtain up-to-date information to control prices on the market and help the customer optimize their own pricing policy.
      Competitive price monitoring software for online reseller allows you to compare prices of online stores selling the same products as your company and work out the best price for your offer. The price parser of competitors for suppliers will allow you to compare the offer of competitors to analogues, and understand how your price is more (or less) attractive to customers.

      Automating price monitoring using a parser has several solutions.

      1. Buying a ready-made parser for price monitoring.
      2. Development of own parser.
      3. Cloud services for monitoring competitors' prices.

      Let's briefly consider the possibilities of each of these methods.

      Buying a ready-made parser based on tables.

      If the issue of monitoring competitors' prices has become acute and there is no time to develop your own, you can buy a ready-made parser and customize it for your own tasks.
      The monitoring results are collected by these parsers in Excel tables and Google sheets.
      The action of these parsers is based on macros of automated actions or special formulas that allow you to collect data from sites.
      This type of parser is suitable for solving simple tasks for collecting data on resources that are not subject to frequent changes. Macros need to be customized. Output results in tabular format only.

      Development of own parser.

      Writing a parser to solve your own problems is more preferable than buying a ready-made one. But there are also nuances here. The first is who to involve in writing a parser for monitoring prices: a freelancer or a company that provides such services? The second nuance is who will maintain and regularly reconfigure the competitor's price parser in case of changes in the layout of online store sites or protection against parsing?
      (For more on scraping issues, see 9 Barriers to Consider When Scraping Competitor Prices from Websites.)

      Payment for the services of customizable competitor price monitoring services.

      Customizable services typically offer turnkey solutions. A clear advantage of such competitor price monitoring services is the specially developed software for collecting data from even the most complex and secure sites. In addition, such services regularly track changes in the settings of sites that are included in the monitoring and reconfigure the parser to collect up-to-date information. The cost of services of such services is much lower compared to the costs, for example, of maintaining your own programmer, servers, etc. or paying manual labor to employees.

      For example, the Price Control service offers solutions not only for standard price monitoring, but also the development of an individual solution for the client's business tasks. The service "Monitoring prices of Marketplaces" has been especially popular lately. This is due to the rapid development of Marketplaces - as a priority for many sales channels on the Internet.

      Collection of prices from supplier price lists

      automatic monitoring of competitors' pricesIn conclusion of the conversation about the possibilities of automated price monitoring, I want to talk about the procedure for working with supplier price lists.
      Supplier price monitoring is a process of regular systematic collection and subsequent analysis of all price offers of online store suppliers: special and promotional offers, discounts and bonus programs to find the minimum purchase price for similar products.
      Each online store has dozens, hundreds, and sometimes thousands of suppliers of goods for sale. It is not uncommon for a store to purchase the same product from several distributors. They compete with each other, trying to offer stores the most favorable conditions for the purchase. It is important for an online store to constantly monitor prices and availability of goods from each supplier in order to determine where to buy goods today is most profitable. It is important for the buyer of an online store to see the full picture of the suppliers' offer in a form convenient for analysis and decision making. It is quite difficult, and often impossible, to solve this problem manually or with the help of price parsers. To solve it, it is more profitable to use the products of services that offer a full range of price monitoring of competitors and suppliers. The Price Control project has good experience in this matter.

      What lawyers say about monitoring

      In this section, I want to say a few words about the legality of data parsing and brand protection on the Internet. Lawyers argue that it is worth paying attention to the legislation on information. As a rule, the legislation of many countries defines information as an object of public, civil and other legal relations. Information can be freely used by any person and transmitted by one person and or other requirements for the procedure for its provision or distribution. Based on the foregoing, information, depending on the category of access to it, is divided into public information, as well as information, access to which is limited by federal laws (restricted access information). Further, the law states that information, depending on the procedure for its provision or distribution, is divided into: information freely distributed, information provided by agreement of persons participating in the relevant relations, information that is subject to provision or distribution, information, the distribution of which is limited or prohibited.
      In most countries, there is no law restricting access to price information in stores. Thus, this information is publicly available, which means that it is not prohibited to collect prices in the online store.
      The law allows everything that is not prohibited. Automatic price monitoring will be legal if it does not violate the prohibitions established by law. This means that when parsing information from Internet sites, it is necessary to comply with current legislation.
      There are a number of restrictions regarding the parsing of information from sites and the protection of intellectual property on the Internet: you cannot violate copyright and related rights, you cannot collect legally protected information, you cannot collect data constituting a trade secret.
      Based on the principle, everything that is not prohibited is allowed, automatic collection of information (web scraping) is not illegal

      • if information is in the public domain and is not protected by copyright and related rights.
      • • Parsing is carried out in compliance with the law.
        • Price monitoring does not disrupt the site.

      When automatically collecting information from sites, it is important:

      1. Do not collect copyrighted content.
      2. Do not disrupt the site during parsing.
      3. Do not violate the terms of use of the site.
      4. Do not collect personal information of users..

      price monitoring in online storesSometimes there are complaints from sites that are exposed to automatic parsing. They argue that the work of parsers interferes with the operation of the site, increases the load on the server, and leads to additional costs for maintaining the site's performance.
      This is done both by Google and Yandex, which are engaged in regular parsing (indexing) of the entire site, and by the mass of competitors who analyze other people's content, by brand reputation monitors, and by potential and real customers of the site ... Prove that it is the parser, collecting price from the site, interferes with its work will be difficult. Services that professionally provide price monitoring services are able to configure their robots so that they have a minimal impact on the operation of the site and do not fall under its protective sanctions.

      Conclusions and recommendations

      I hope I managed to convince the reader in this article that specialized services will be the best price control tool on the market. Only they will give complete information, bypass all site protections, save you a lot of time and money. Now let's talk about what the price of their services depends on.
      It depends on the amount of data collected. There is a difference between collecting only the prices of goods from websites and collecting data on price, availability, promotions, discounts, shipping, credit ...
      The price also depends on the data presentation format. It is one thing to simply display monitoring data in a table, and it is quite another to provide the client with the opportunity to work in a convenient personal account. There, the client can see all the data in dynamics, build visual graphs, select the information he is interested in using numerous filters, set up the distribution of information, and much more.

      Standard price monitoring table.

      price monitoring ukrainePresentation of data in a personal account.

      The biggest impact on the cost of the monitoring service is the responsibility for matching the client's goods with the goods on the websites of Internet resources. If the client independently compares, the price of monitoring for him is much lower. If the service is fully responsible for matching when setting up the project and regularly searches for new products on the sites it monitors, the price will be more expensive.

      You can read about the successful experience of cooperation with us in our blog section our cases.

      Price Control - a time-tested program for monitoring prices in online stores

      magnifying glassOur price monitoring service Price Control has been helping clients earn more for the seven year. Therefore, if you need a comprehensive solution for the development of a company (from price monitoring to a full range of anti-dumping measures), then we advise you to contact the professionals.

      Our program for monitoring prices in online stores, our experience and knowledge will help you to use the full potential of your company and move to a qualitatively new level of business. We will be sincerely glad if you choose our service Price controls.

      Want to learn more or book a consultation with a specialist? Contact us!

        Сергей Кокоулин
        Article author: Marketing - Project Director at PriceControl.

        How useful was this post?

        Click on a star to rate it!

        Average rating 4.5 / 5. Vote count: 15

        No votes so far! Be the first to rate this post.

         Share on social networks

        leave a comment

        We are in social networks