What this article covers
- Explains how the AI Classifier Feedback tool enables users to provide feedback on classifier results and how it supports collaboration between reviewers and classifier builders.
- Describes how users can submit positive or negative feedback on patent family classifications and view feedback history based on access rights.
- Outlines how feedback is reviewed by classifier owners, how email notifications work, and how administrators manage access permissions.
What is the AI Classifier feedback tool?
Where can I find the AI Classifier feedback tool?
How can I see which AI Classifiers I have access to?
How to provide AI Classifier feedback
Understanding the AI Classifier Feedback card
View the Feedback History of a Patent Family
How to review Classifier feedback (Builder)
How to enable the AI Classifier feedback function (Administrators)
What is the AI Classifier feedback tool?
The AI Classifier Feedback tool allows reviewers to provide expert feedback on classifier results directly within the LexisNexis® PatentSight+™ platform during the classifier build and evaluation process.
From the Patent Family card in the Result List, reviewers can see which classifiers a selected patent family belongs to (depending on their access rights). They can then either:
Agree with the classification (positive match), or
Disagree with the classification (negative match).
This enables IP professionals to collaborate with classifier builders by contributing domain expertise to help improve classification precision.
Handy Hint: Submitting feedback does not automatically retrain or modify the classifier. Feedback is sent to the classifier owner via email, which remains the primary method for receiving notifications. Visibility of feedback within the platform depends on user access rights (see below).
Where can I find the AI Classifier feedback tool?
The classifier feedback tool can be found in the Patent Family card view of the Result List:
Reviewers in PatentSight who have access to the AI Classifier Feedback feature can:
Mark a patent family as Negative if it is currently included in the classifier but falls outside the intended technology scope.
Mark a patent family as Positive if it should be included in the classifier but is not currently classified as such.
How can I see which AI Classifiers I have access to?
As a reviewer, you can see which AI Classifiers you have access to by navigating to Options > Manage Access for AI Classifiers from the toolbar:
Note - this section will not tell you if you have specific 'feedback' access to an AI classifier. From here, you can see which AI Classifiers you have 'read' access to, or 'feedback' access.
For more information on AI Classifier access rights, click here.
How to provide AI Classifier feedback
You would like to provide feedback to the builder on a classifier. There are two main use cases for the classifier feedback function:
Mark a Patent Family as Negative
Use this option if a patent family is currently included in the classifier results but should not be, because it falls outside the intended technology scope:
- First, run your report applying the classifier.
- Then from the 'Analysis Templates' section, click 'Result List' to access the list of patent families currently determined as 'Positive' to the classifier.
- In the Results List, select the patent family from the list that you want to mark as 'Negative' to the classifier:
- In the Patent Family card, select Negative in the AI Classifier section (round icon with an 'x' next to the AI Classifier you want to mark as a negative). You can add a comment for the builder if needed. The classifier will already appear here because you applied it when generating the report.
- Click 'Submit Feedback'. The builder will receive this feedback in an email.
Note - Your name will be shared with the classifier owner along with your feedback. Your name will also be shared in the 'Feedback History' section.
Handy Hint: If there is a classifier in your account for which you believe this patent family IS relevant, you can easily add the classifier here to provide positive feedback:
- Select the 'plus +' icon next to 'AI Classifier'. Select the classifier that you want to mark the patent family as 'Positive' for. It will then appear in the Patent Family card, and you can not only mark as 'positive' but you can also provide a comment to the builder if needed. Select 'Submit Feedback'.
Summary - When selecting the classifier in the Patent Family card, you are either agreeing (Positive) or disagreeing (Negative) with the classifier’s current assignment of that patent family. The builder will receive this feedback in an email.
Mark a Patent Family as Positive
Use this option if a patent family is not currently included in the classifier results but should be, because it fits within the intended technology scope.
- Access the 'Analysis Templates' section of your Workbook, and click 'Result List' to access the list of patent families you want to feedback on.
Note - that this doesn't have to be a search with AI Classifiers; it can be any search type but the search should include the patent families you want to provide feedback on, to the Classifier builder.)
- In the results list, select the patent family that you want to provide feedback for. Scroll down the patent family card to the 'AI Classifier' feedback section:
- If the required classifier for feedback is not shown in the list, select the plus icon next to 'AI Classifiers' to add it in:
- Select 'Positive' next to the classifier, add a comment for the builder if required and select 'Submit Feedback':
Understanding the AI Classifier Feedback card
The total count of positive feedback submitted for the AI Classifier for this patent family. |
|
| The total count of negative feedback submitted for the AI Classifier for this patent family. | |
| This AI Classifier is suggested as part of the patent family. To provide feedback, select relevant or irrelevant and submit feedback. | |
| You have submitted feedback for this AI Classifier. | |
| View the feedback history of the AI Classifiers that you have access to, for this patent family |
View the Feedback History of a Patent Family
You can view the feedback history for any AI Classifier you have access to at patent family level.
From the Result List, open the Patent Family card and select Feedback History in the AI Classifier section.
This opens a separate window displaying all feedback associated with that patent family for the classifiers you have permission to access. The feedback history for the selected patent family shows:
The AI Classifier name the feedback relates to
Whether the feedback was Positive or Negative
Who provided the feedback
The date the feedback was submitted
When the classifier was last built
Only feedback for classifiers you have access to will be visible.
How to review Classifier feedback - Builder
When the feedback is sent, the users who are classifier owners will receive an email. This email will contain all feedback entries for the respective classifier for the last day.
If the user owns multiple classifiers, there will be a separate email per classifier.
The emails are sent from alerts@patentsight.com.
The respective feedback (Positive/Negative) is shown with a cross or a tick in the feedback column of the email, and the comment is also displayed if one has been made by the reviewer.
The builder can also access the classifier directly from this email in order to action any of the feedback received.
The email is currently the primary (and at present sole) method for receiving classifier feedback. In-platform display of feedback is being developed but is not yet fully available.
The feedback email will now include the name of the user who provided the feedback (previously feedback was anonymous).
Please reach out to your customer success manager, who will be happy to help you, or send us a note at support@lexisnexisip.com if you'd like to learn more.
How to enable the AI Classifier feedback function (Administrators)
The AI Classifier Feedback functionality is controlled by your PatentSight+ Administrator. If enabled for a PatentSight+ user, they can then provide AI Classifier feedback via the Result List. If the feature package is removed, users will not be able to provide feedback at all.
Access is controlled from Options > Manage Access for AI Classifiers section:
From here, Administrators can grant access to AI Classifiers, and as a reviewer, you can see which AI Classifiers you have access to also.
When sharing a classifier (or taxonomy node), users can be granted:
Admin access – full control and visibility of feedback history
Feedback-level access – can provide feedback and view feedback history
Read-only access – cannot provide feedback or view feedback history
Users will only be able to see feedback history if they have Feedback-level access or above. More information on access permissions can be found here.
If you'd like to know more, or would like to use this feature, please reach out to your customer success manager, who will be happy to help you, or send us a note at support@lexisnexisip.com.