This guide offers a comprehensive approach to generating detailed reports and interpreting data from your HR processes. By assessing the completeness and accuracy of your profiles, you can ensure higher quality candidate data, leading to more effective recruitment and talent management.

A. Why Evaluate Parsing Quality?

Understanding the quality of parsing models to accurately detect and fill various fields in a profile is crucial for several reasons:

• Assess Model Performance: Determine which HrFlow.ai parsing model (Quicksilver, Hawk, Mozart) best fits your specific use case.
• Benchmarking: Compare HrFlow.ai's models against competitors to make informed decisions.

B. Step by steps

B.1. Initialize the HrFlow Client

📘

Prerequisites

• ✨ Create a Workspace
• 🔑 Get your API Key
• HrFlow.ai Python SDK version 4.0.0 or above: Install it via pip with pip install -U hrflow>=4.0.0 or conda with conda install hrflow>=4.0.0 -c conda-forge.

First, initialize the HrFlow client with your API credentials.

from hrflow import Hrflow
from hrflow.utils import generate_parsing_evaluation_report

client = Hrflow(api_secret="your_api_secret", api_user="your_api_user")

B.2. Ensure Your Data is Ready

Before generating the parsing evaluation report, you need to ensure that your data is ready. This involves parsing resumes in a source.

1. 🧠 Activate Profile Parsing API
2. 🔌 Create a Source

The function used for this purpose is add_folder, which parses a folder of profile resumes to a specified source key.

❗️

Attention, if you use asynchronous parsing

if you use asynchronous parsing, the CVs may take some time to be parsed. Therefore, make sure to wait until all the CVs are parsed and present in the source before moving on to the next step.

Make sure you have created a source and parsed profiles in this source. For more details on creating a source, refer to the Connectors Source Documentation.

import os

STORAGE_DIRECTORY_PATH = "./resumes"  # Path where your resumes are stored
FAILURES_DIRECTORY_PATH = "./failures"  # Path to store failed parses

os.makedirs(STORAGE_DIRECTORY_PATH, exist_ok=True)
os.makedirs(FAILURES_DIRECTORY_PATH, exist_ok=True)

results = client.profile.parsing.add_folder(
    source_key=SOURCE_KEY,
    dir_path=STORAGE_DIRECTORY_PATH,
    is_recursive=True,
    move_failure_to=FAILURES_DIRECTORY_PATH,
    show_progress=True,
    max_requests_per_minute=30,
    min_sleep_per_request=1,
)

Field Explanations:

B.3. Generate the Parsing Evaluation Report

Use the generate_parsing_evaluation_report function to create the evaluation report.

generate_parsing_evaluation_report(
    client,
    source_key="YOUR_SOURCE_KEY",
    report_path="parsing-evaluation.xlsx",
    show_progress=True,
)

Field Explanations:

B.4 Interpreting the Results

The generated report is an Excel file named parsing-evaluation.xlsx that provides a detailed assessment of the quality of parsing models to detect and fill various fields in profiles stored in the specified HrFlow source.

The report consists of two sheets:

1. Definition: This sheet explains each field and how to interpret the results.

   

   1. Score : Represents the filling rate of a given field (e.g. 0.98 in Experiences/title means 98% of the experiences contains a title)
   2. Existence : Either 0 or 1, check wether a field has been found (e.g. 1 in Informations/phone means a phone number have been found)
   3. Count : Counts the number of element of a given type have been found (e.g. 2 in Experiences/skills means 2 skills have been found among all the experiences)

2. Statistics: This sheet presents a comprehensive set of results, offering insights into parsing quality across various sections such as personal info, experience, education, and other skills.

   

   1. General Information:
      1. filename: The name of the file containing the profile.
      2. resume: A link to the resume.
      3. profile: A link to the parsed profile JSON in HrFlow.ai.
   2. Information:
      1. score: Overall score for personal information. It's an average on all personal info fields.
      2. person, phone, email, location, summary, driving_license: Scores indicating the presence and completeness of each type of information.
   3. Experiences:
      1. score: Overall score for the experience section. It's an average on all experience score fields.
      2. count: The number of experiences detected.
      3. title, company, start_date, end_date, location, desc: Scores for the presence and completeness of each field within experiences.
      4. skills, tasks, courses, certifications: Counts of additional elements found within experiences.
   4. Educations:
      1. score: Overall score for the education section. It's an average on all experience score fields.
      2. count: The number of education entries detected.
      3. title, school, start_date, end_date, location, desc: Scores for the presence and completeness of each field within education.
      4. skills, tasks, courses, certifications: Counts of additional elements found within education.
   5. Other Sections:
      1. skills: Total number of skills detected across the entire profile.
      2. languages: Total number of languages detected.
      3. tasks: Total number of tasks detected.
      4. courses: Total number of courses detected.
      5. certifications: Total number of certifications detected.
      6. interests: Total number of interests detected.

Key Points to Note

• Scores Reflect Completeness: A low score may indicate missing information in the resumes rather than poor parsing performance.
• Context Matters: For anonymized resumes, expect lower scores in personal information fields due to the absence of such data.

D. Additional Resources

• Feature Announcement
• HrFlow.ai Python SDK on PyPI
• HrFlow Cookbook : Repository containing helpful notebooks by the HrFlow.ai team
• Parsing Evaluator Notebook in Colab