Project Background

ESM, an internet marketing agency, faced significant challenges in automating the creation of tare sheets for their marketing campaigns. Tare sheets, which are demo ad pages, display how ads would appear on different websites. The agency wanted to streamline this process, reducing the need for manual intervention while ensuring that the ad creatives could be quickly previewed on various web pages. Their goal was to eliminate the repetitive and time-consuming manual work involved in uploading creatives and replacing ads, thus enhancing workflow efficiency.

Challenges

The primary challenges faced by ESM were the manual and repetitive nature of creating tare sheets, which required substantial human involvement for tasks such as uploading creative materials and replacing ads on different websites. This process was time-intensive and hindered scalability. Additionally, segmenting and identifying ads accurately on websites was not straightforward, and the client had limited resources and technology to automate these tasks effectively. These constraints made it difficult for ESM to scale their marketing efforts efficiently.

Solution

To solve these challenges, an automated system was developed that could scrape websites, identify where ads were located, and replace them with the provided ad creatives. The solution utilized the Flash API for capturing screenshots of the websites, while custom OpenCV code was used to detect bounding boxes around images to identify ads. Additionally, GPT-4O was integrated to help classify and identify the ads on the web pages. The system was built using the Agency Swarm framework, which managed the web scraping, ad detection, and replacement tasks. This solution allowed the tare sheet creation process to be automated, reducing human involvement significantly while ensuring accuracy and efficiency.

Key Features and Functionality

• AI Agents Developed:
  • Web Scraping Agent: Captures images of websites.
  • Ad Replacement Agent: Detects and replaces ads with the client’s ad creatives using custom object detection.
• Integrations: API Flash for web scraping was integrated into the system for capturing screenshots.
• Customization: Custom OpenCV code was implemented to identify the bounding boxes of images to pinpoint ads, which was essential for the accurate replacement of ads with client-provided creatives.

Results and Impact

The implementation of this AI-powered system successfully automated the tare sheet creation process, allowing ESM to quickly preview their ads on various websites. The system was able to accurately replace ads on high-profile websites like The New York Times, Forbes, and TechCrunch. Furthermore, the solution proved to be highly cost-effective, with the cost per tare sheet being approximately three to five cents. Client feedback was extremely positive, highlighting how the automation tool drastically reduced the time spent on tare sheet creation and allowed the agency to focus more on creative strategies.

Lessons Learned

The project’s success highlighted the effectiveness of integrating AI models like YOLO for ad detection and replacement, as well as the use of custom OpenCV code to improve accuracy. One of the key challenges overcome was identifying which images on a webpage were ads, which was achieved through a combination of OpenCV and GPT-4O. Moving forward, there are plans to further refine the model by using OpenCV more extensively for ad detection, potentially reducing reliance on third-party models. This could lead to even greater efficiency and scalability in future iterations of the solution.

Conclusion

The AI-powered solution successfully automated the tare sheet creation process, improving efficiency for ESM and reducing manual work. This system is scalable and cost-effective, and future plans include refining the model for even greater accuracy and expanding its use for other types of marketing campaign automation.

Future plans involve enhancing the solution's capabilities and possibly expanding it to other industries for similar use cases.