
file = '/home/staccioli/data/patentsview/tsv/detail_desc_text_2019.tsv'


import pandas as pd

df = pd.read_csv(file,
                 delimiter='\t',
                 dtype={'uuid': 'object',
                        'patent_id': 'object',
                        'text': 'object',
                        'length': 'int64'})

df


desc = df.loc[1391]['text']


print(desc[:1000])

Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

FIG. 1shows a container20comprising the assembly of a bottle body22, a spout fitment24(FIG. 6), and a cap26(which may serve as a measuring/dispensing cup). Each may be made as a unitary plastic molding. Exemplary bottle body material is high density polyethylene (HDPE). Exemplary spout fitment and cap material is polypropylene (e.g., LDPE for the spout fitment and copolymer polypropylene for the cap).

The body22comprises a unitary combination of a base30, a sidewall32extending upward from the base, a shoulder34at an upper end of the sidewall, and a neck36extending upward from the shoulder to a rim38(FIG. 4A) and defining an opening40(FIG. 9) having a central longitudinal axis500. In this example, an axis500(FIG. 3) serves as a central longitudinal axis of the spout fitment, cap, and neck of the bottle. In this exemplary embodiment, the axis500is vertical when the bottle is re


import re

desc = re.sub('\d', ' ', desc)


print(desc[:1000])

Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

FIG.  shows a container  comprising the assembly of a bottle body  , a spout fitment  (FIG.  ), and a cap  (which may serve as a measuring/dispensing cup). Each may be made as a unitary plastic molding. Exemplary bottle body material is high density polyethylene (HDPE). Exemplary spout fitment and cap material is polypropylene (e.g., LDPE for the spout fitment and copolymer polypropylene for the cap).

The body  comprises a unitary combination of a base  , a sidewall  extending upward from the base, a shoulder  at an upper end of the sidewall, and a neck  extending upward from the shoulder to a rim  (FIG.  A) and defining an opening  (FIG.  ) having a central longitudinal axis   . In this example, an axis   (FIG.  ) serves as a central longitudinal axis of the spout fitment, cap, and neck of the bottle. In this exemplary embodiment, the axis   is vertical when the bottle is re


desc = desc.lower()


print(desc[:1000])

like reference numbers and designations in the various drawings indicate like elements.

detailed description

fig.  shows a container  comprising the assembly of a bottle body  , a spout fitment  (fig.  ), and a cap  (which may serve as a measuring/dispensing cup). each may be made as a unitary plastic molding. exemplary bottle body material is high density polyethylene (hdpe). exemplary spout fitment and cap material is polypropylene (e.g., ldpe for the spout fitment and copolymer polypropylene for the cap).

the body  comprises a unitary combination of a base  , a sidewall  extending upward from the base, a shoulder  at an upper end of the sidewall, and a neck  extending upward from the shoulder to a rim  (fig.  a) and defining an opening  (fig.  ) having a central longitudinal axis   . in this example, an axis   (fig.  ) serves as a central longitudinal axis of the spout fitment, cap, and neck of the bottle. in this exemplary embodiment, the axis   is vertical when the bottle is re


desc = desc.replace('fig.', '')


print(desc[:1000])

like reference numbers and designations in the various drawings indicate like elements.

detailed description

  shows a container  comprising the assembly of a bottle body  , a spout fitment  (  ), and a cap  (which may serve as a measuring/dispensing cup). each may be made as a unitary plastic molding. exemplary bottle body material is high density polyethylene (hdpe). exemplary spout fitment and cap material is polypropylene (e.g., ldpe for the spout fitment and copolymer polypropylene for the cap).

the body  comprises a unitary combination of a base  , a sidewall  extending upward from the base, a shoulder  at an upper end of the sidewall, and a neck  extending upward from the shoulder to a rim  (  a) and defining an opening  (  ) having a central longitudinal axis   . in this example, an axis   (  ) serves as a central longitudinal axis of the spout fitment, cap, and neck of the bottle. in this exemplary embodiment, the axis   is vertical when the bottle is resting atop a horizon


from nltk.tokenize import sent_tokenize

sentences = sent_tokenize(desc)


sentences[:10]

['like reference numbers and designations in the various drawings indicate like elements.',
 'detailed description\n\n  shows a container  comprising the assembly of a bottle body  , a spout fitment  (  ), and a cap  (which may serve as a measuring/dispensing cup).',
 'each may be made as a unitary plastic molding.',
 'exemplary bottle body material is high density polyethylene (hdpe).',
 'exemplary spout fitment and cap material is polypropylene (e.g., ldpe for the spout fitment and copolymer polypropylene for the cap).',
 'the body  comprises a unitary combination of a base  , a sidewall  extending upward from the base, a shoulder  at an upper end of the sidewall, and a neck  extending upward from the shoulder to a rim  (  a) and defining an opening  (  ) having a central longitudinal axis   .',
 'in this example, an axis   (  ) serves as a central longitudinal axis of the spout fitment, cap, and neck of the bottle.',
 'in this exemplary embodiment, the axis   is vertical when the bottle is resting atop a horizontal surface.',
 'other orientations may be possible.',
 'with this exemplary frame of reference, with the bottle in the upright condition, a direction   (  ) is upward parallel to the axis   and in opposite direction   is downward.']


from nltk.tokenize import word_tokenize

tokens = [word_tokenize(sentence) for sentence in sentences]


tokens[:2]

[['like',
  'reference',
  'numbers',
  'and',
  'designations',
  'in',
  'the',
  'various',
  'drawings',
  'indicate',
  'like',
  'elements',
  '.'],
 ['detailed',
  'description',
  'shows',
  'a',
  'container',
  'comprising',
  'the',
  'assembly',
  'of',
  'a',
  'bottle',
  'body',
  ',',
  'a',
  'spout',
  'fitment',
  '(',
  ')',
  ',',
  'and',
  'a',
  'cap',
  '(',
  'which',
  'may',
  'serve',
  'as',
  'a',
  'measuring/dispensing',
  'cup',
  ')',
  '.']]


from nltk.corpus import stopwords


stops = set(stopwords.words('english'))
sorted(list(stops))[:15]

['a',
 'about',
 'above',
 'after',
 'again',
 'against',
 'ain',
 'all',
 'am',
 'an',
 'and',
 'any',
 'are',
 'aren',
 "aren't"]


tokens = [[token for token in sentence if token not in stops] for sentence in tokens]


tokens[:2]

[['like',
  'reference',
  'numbers',
  'designations',
  'various',
  'drawings',
  'indicate',
  'like',
  'elements',
  '.'],
 ['detailed',
  'description',
  'shows',
  'container',
  'comprising',
  'assembly',
  'bottle',
  'body',
  ',',
  'spout',
  'fitment',
  '(',
  ')',
  ',',
  'cap',
  '(',
  'may',
  'serve',
  'measuring/dispensing',
  'cup',
  ')',
  '.']]


from nltk.stem.porter import PorterStemmer
stemmer = PorterStemmer()


stems = [[stemmer.stem(token) for token in sentence] for sentence in tokens]


stems[:2]

[['like',
  'refer',
  'number',
  'design',
  'variou',
  'draw',
  'indic',
  'like',
  'element',
  '.'],
 ['detail',
  'descript',
  'show',
  'contain',
  'compris',
  'assembl',
  'bottl',
  'bodi',
  ',',
  'spout',
  'fitment',
  '(',
  ')',
  ',',
  'cap',
  '(',
  'may',
  'serv',
  'measuring/dispens',
  'cup',
  ')',
  '.']]


predicates = ['reduce', 'replace', 'eliminate', 'save', 'lower', 'substitute', 'automate', 'less']
objects = ['worker', 'labor', 'labour', 'human', 'employee', 'manpower', 'job', 'manual']
attributes = ['cost', 'expenditure', 'expens', 'hour', 'intensity', 'task', 'time', 'skill', 'effort', 'intervention']


predicates = [stemmer.stem(word) for word in predicates]
objects = [stemmer.stem(word) for word in objects]
attributes = [stemmer.stem(word) for word in attributes]


print(predicates)
print(objects)
print(attributes)

['reduc', 'replac', 'elimin', 'save', 'lower', 'substitut', 'autom', 'less']
['worker', 'labor', 'labour', 'human', 'employe', 'manpow', 'job', 'manual']
['cost', 'expenditur', 'expen', 'hour', 'intens', 'task', 'time', 'skill', 'effort', 'intervent']


from itertools import product

triplets = list(product(predicates, objects, attributes))


triplets[:20]

[('reduc', 'worker', 'cost'),
 ('reduc', 'worker', 'expenditur'),
 ('reduc', 'worker', 'expen'),
 ('reduc', 'worker', 'hour'),
 ('reduc', 'worker', 'intens'),
 ('reduc', 'worker', 'task'),
 ('reduc', 'worker', 'time'),
 ('reduc', 'worker', 'skill'),
 ('reduc', 'worker', 'effort'),
 ('reduc', 'worker', 'intervent'),
 ('reduc', 'labor', 'cost'),
 ('reduc', 'labor', 'expenditur'),
 ('reduc', 'labor', 'expen'),
 ('reduc', 'labor', 'hour'),
 ('reduc', 'labor', 'intens'),
 ('reduc', 'labor', 'task'),
 ('reduc', 'labor', 'time'),
 ('reduc', 'labor', 'skill'),
 ('reduc', 'labor', 'effort'),
 ('reduc', 'labor', 'intervent')]


for i, sentence in enumerate(stems):
    if any(all(word in sentence for word in words) for words in triplets):
        print(sentences[i])

it may also avoid the need for more complicated spout fitment arrangements to provide good sealing (and thereby resulting in labor and/or material cost savings).


def is_LS(description):
    text = re.sub('\d', ' ', description)
    text = text.lower()
    text = text.replace('fig.', '')
    sentences = sent_tokenize(text)
    tokens = [word_tokenize(sentence) for sentence in sentences]
    tokens = [[token for token in sentence if token not in stops] for sentence in tokens]
    stems = [[stemmer.stem(token) for token in sentence] for sentence in tokens]
    for i, sentence in enumerate(stems):
        if any(all(word in sentence for word in words) for words in triplets):
            return(sentences[i])
    return(False)


df_slice = df[1380:1480].copy()


df_slice


from tqdm.notebook import tqdm
tqdm.pandas()


df_slice['LS'] = df_slice['text'].progress_apply(is_LS)


df_slice


df_slice[df_slice['LS'] != False]


print(df_slice.loc[1391]['LS'])

it may also avoid the need for more complicated spout fitment arrangements to provide good sealing (and thereby resulting in labor and/or material cost savings).


print(df_slice.loc[1412]['LS'])

employers will like the cost savings and more productive employees.


print(df_slice.loc[1479]['LS'])

in addition, the process of recharging the assembly (i.e., removing spent resin from the vessel and replacing it with functional resin) with head  attached to the vessel is inefficient, usually requiring human intervention to remove the spent resin.

	uuid	patent_id	text	length
0	ec3d68a0-fb70-11ea-a344-121df0c29c1e	10165721	DESCRIPTION OF THE PREFERRED EMBODIMENTS\n\nWi...	2626
1	ec3e1e7c-fb70-11ea-a344-121df0c29c1e	10165722	DETAILED DESCRIPTION\n\nHuman beings have cert...	38836
2	ec3e77cf-fb70-11ea-a344-121df0c29c1e	10165723	DETAILED DESCRIPTION\n\nReferring toFIGS. 1, 1...	10803
3	ec3f78f2-fb70-11ea-a344-121df0c29c1e	10165724	DETAILED DESCRIPTION OF THE PREFERRED EMBODIME...	13949
4	ec401255-fb70-11ea-a344-121df0c29c1e	10165725	DETAILED DESCRIPTION\n\nBefore describing the ...	71115
...	...	...	...	...
357785	51ca1f22-073f-11eb-a344-121df0c29c1e	RE47790	DETAILED DESCRIPTION OF THE INVENTION\n\nThe f...	12328
357786	51ca22ed-073f-11eb-a344-121df0c29c1e	RE47791	DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS\...	21338
357787	51ca338f-073f-11eb-a344-121df0c29c1e	RE47792	DETAILED DESCRIPTION OF THE INVENTION\n\nRefer...	175926
357788	51ca3fa5-073f-11eb-a344-121df0c29c1e	RE47793	DESCRIPTION OF EXAMPLE EMBODIMENTS OF THE PRES...	48469
357789	51ca45a5-073f-11eb-a344-121df0c29c1e	RE47794	BEST MODE FOR CARRYING OUT THE INVENTION\n\nPr...	34116

	uuid	patent_id	text	length
1380	f211b53a-fb70-11ea-a344-121df0c29c1e	10167107	DETAILED DESCRIPTION\n\nReferring firstly toFI...	25907
1381	f211d8b5-fb70-11ea-a344-121df0c29c1e	10167108	DETAILED DESCRIPTION\n\nFIG. 1illustrates a sh...	9874
1382	f211f88c-fb70-11ea-a344-121df0c29c1e	10167109	To facilitate an understanding of the embodime...	12768
1383	f2121bd9-fb70-11ea-a344-121df0c29c1e	10167110	DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT...	14073
1384	f212b6f1-fb70-11ea-a344-121df0c29c1e	10167111	As will be appreciated in light of this disclo...	71223
...	...	...	...	...
1475	f25a2bff-fb70-11ea-a344-121df0c29c1e	10167202	DETAILED DESCRIPTION\n\nIntroduction\n\nThe va...	32666
1476	f25a95ed-fb70-11ea-a344-121df0c29c1e	10167203	DETAILED DESCRIPTION OF THE DRAWINGS\n\nIt wil...	28884
1477	f25aee2c-fb70-11ea-a344-121df0c29c1e	10167204	The embodiments set forth in the drawings are ...	26020
1478	f26520e3-fb70-11ea-a344-121df0c29c1e	10167205	DETAILED DESCRIPTION\n\nVarious embodiments of...	51869
1479	f26605d3-fb70-11ea-a344-121df0c29c1e	10167206	DETAILED DESCRIPTION\n\nDeionization of water ...	34129

	uuid	patent_id	text	length	LS
1380	f211b53a-fb70-11ea-a344-121df0c29c1e	10167107	DETAILED DESCRIPTION\n\nReferring firstly toFI...	25907	False
1381	f211d8b5-fb70-11ea-a344-121df0c29c1e	10167108	DETAILED DESCRIPTION\n\nFIG. 1illustrates a sh...	9874	False
1382	f211f88c-fb70-11ea-a344-121df0c29c1e	10167109	To facilitate an understanding of the embodime...	12768	False
1383	f2121bd9-fb70-11ea-a344-121df0c29c1e	10167110	DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT...	14073	False
1384	f212b6f1-fb70-11ea-a344-121df0c29c1e	10167111	As will be appreciated in light of this disclo...	71223	False
...	...	...	...	...	...
1475	f25a2bff-fb70-11ea-a344-121df0c29c1e	10167202	DETAILED DESCRIPTION\n\nIntroduction\n\nThe va...	32666	False
1476	f25a95ed-fb70-11ea-a344-121df0c29c1e	10167203	DETAILED DESCRIPTION OF THE DRAWINGS\n\nIt wil...	28884	False
1477	f25aee2c-fb70-11ea-a344-121df0c29c1e	10167204	The embodiments set forth in the drawings are ...	26020	False
1478	f26520e3-fb70-11ea-a344-121df0c29c1e	10167205	DETAILED DESCRIPTION\n\nVarious embodiments of...	51869	False
1479	f26605d3-fb70-11ea-a344-121df0c29c1e	10167206	DETAILED DESCRIPTION\n\nDeionization of water ...	34129	in addition, the process of recharging the ass...

	uuid	patent_id	text	length	LS
1391	f2148665-fb70-11ea-a344-121df0c29c1e	10167118	Like reference numbers and designations in the...	13911	it may also avoid the need for more complicate...
1412	f2195a0e-fb70-11ea-a344-121df0c29c1e	10167139	DETAILED DESCRIPTION OF THE INVENTION\n\nThis ...	27521	employers will like the cost savings and more ...
1479	f26605d3-fb70-11ea-a344-121df0c29c1e	10167206	DETAILED DESCRIPTION\n\nDeionization of water ...	34129	in addition, the process of recharging the ass...

Specify target file¶

Import tsv file as pandas dataframe¶

EXAMPLE¶

Select one for the sake of example¶

Remove digit characters (often misplaced footnote markers)¶

Toggle lowercase¶

Remove ''noisy'' strings¶

Tokenise by sentence¶

Tokenise by word¶

Remove stop words¶

Stem/Lemmatise tokens¶

Query definition¶

Query execution¶

GENERALISATION¶

Define unified function¶

Slice the original data frame¶

Add a new column with matched sentence, if LS, or False otherwise¶