Crawlers for large amounts of data

Challenge

As Steven Johnson wrote in his book “Where Good Ideas Come: A Brief History of Innovation,” it is precisely where many people come together that most innovations arise. For this reason, many companies regularly visit trade fairs and congresses in order to get to know exhibitors in person and to include ideas that they have not previously had in the spotlight. It doesn’t matter if you want to open up new business areas, expand the existing customer base, look for new employees or look for new technologies with which you can simplify and improve your own work. The meeting of many creates a potential that must be exploited. However, in order to find your way around the event, it is necessary to learn so much information about the type, participants and topics of the respective event. To make this possible, trade fair or congress operators publish exhibitor, topics or lecture lists. These are sometimes even provided with rudimentary filter functions to allow interested companies to plan their visit or to decide to visit in the first place. The challenge for companies is to find out which topics are offered where, which exhibitors are interesting and which lectures are offered. The analysis of the required data and its addition with additional information critical of decision-making is fundamental for an efficient course.

Goal

The aim of the project was to develop an intelligent platform that enables the client to prepare, analyse and expand the information provided to him with additional information relevant to him, so that he can use the relevant information for him to information for his visit or he can decide for or against an event visit.

Result

The result of this challenging task is a platform that, freely configurable to the respective corporate goals, automatically searches trade fair pages for appropriate exhibiting, can be selected from the Internet or other Adds data sources to provide the user with decision-making aids for or against a visit, provides contact details for potential contact persons of the individual exhibitors or speakers and thus shows an optimal plan for attending the event.

Example

A globally active group helps other internationally producing companies in Asia to become manufacturer-oriented and wants to expand its customer portfolio. To this end, the Group is looking for all companies with german headquarters that do not yet produce in Asia, have a minimum number of employees and minimum turnover and have a product range that can be used for outsourcing to Asian countries from different reasons such as quantity, price or quality could be interesting.

Services

• Employees: 1-2 for Back-end Development, Front-end Development, Project Management, Quality Assurance
• Duration: 3+ years
• Methodology: Scrum