「同志」怎麼了？

上次我們發現了當「同志」一詞作為複合名詞的前半部時，在兩個語料中有不同的分佈情況，尤其是下福盟的語料當中有些有趣的用法。這次我想看的是當句子出現同志一詞時，所使用的動詞有什麼不同。常說動詞/述詞就像串起句子元素的主線，句子中的許多成分好比該動詞所表達的動作的參與角色。同志既可以作為受事者被霸凌、被忽視、被壓迫，也可以作為主事者站出來、進行抗爭、要求權利，因此我好奇兩個語料中的「同志」是否參與了不同動作，成為不同故事/敘事中的主角。

以下我將練習用 jieba.posseg 所提供的詞性標注功能，針對兩個語料當中，包含「同志」的句子進行詞性標註，並篩選出所有的動詞。

import nltk
import re
import json
import jieba
import jieba.posseg as pseg
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib.font_manager import FontProperties
from IPython.display import display
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.decomposition import TruncatedSVD

jieba.set_dictionary('jieba_dict/dict.txt.big')
jieba.load_userdict('jieba_dict/user_dict.txt')

plt.rcParams['figure.figsize'] = [10, 5]
myfont = FontProperties(fname=r'/usr/local/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/msj.ttf',size=14)
sns.set(font=myfont.get_name())

%matplotlib inline

pd.set_option('display.max_rows', 2000)

LGBTFAMILY_PATH = "raw_content_pro.json"
LOVEFAMILY_PATH = "raw_content_anti.json"

lgbtfamily_raw_text = ""
lovefamily_raw_text = ""

with open(LGBTFAMILY_PATH) as f:
    lgbtfamily_raw_text = json.loads(f.read())

with open(LOVEFAMILY_PATH) as f:
    lovefamily_raw_text = json.loads(f.read())

Building prefix dict from /Users/koukantou/Desktop/Don/ntu_master/107-2-lab-meeting/nano-blog/jieba_dict/dict.txt.big ...
Dumping model to file cache /var/folders/vp/g4gpl3x91qqdn6_x0t8w4bbc0000gn/T/jieba.u336b009ae6d5715d70945d8db9630e20.cache
Loading model cost 2.718 seconds.
Prefix dict has been built succesfully.

# 前處理函式
def generate_line_sentence(all_text):  
    # 以句號分句
    all_text = re.split('[。，！？!?,]', all_text)
    
    # 去除所有非中文字元
    all_text = [_get_chinese(s) for s in all_text]
    
    return all_text

# 接收語料(dict)與關鍵詞(string)，回傳語料當中關鍵詞出現的句子中，按照頻率排列的動詞表
def generate_verb_list(raw_text_list, word):
    a = [generate_line_sentence(article['content']) for article in raw_text_list]
    l = [sen for article in a for sen in article if word in sen]
    l = [list(pseg.cut(ll)) for ll in l]
    wordlist = [word for sen in l for (word, pos) in sen if pos.startswith('v')]
    freq_dict = {w:wordlist.count(w) for w in set(wordlist)}
    return sorted(freq_dict.items(), key=lambda kv: -kv[1])

# 接收語料(dict)與關鍵詞(string)，回傳語料中含有該關鍵詞的concordance dataframe
def generate_concordance_df(raw_text_list, word):
    a = [generate_line_sentence(article['content']) for article in raw_text_list]
    l = [sen.split(word) for article in a for sen in article if word in sen]
    df = pd.DataFrame({
    "left_context": [s[0] for s in l],
    "keyword": word,
    "right_context": [s[1] for s in l],
    })
    return df

伴侶盟中的同志

先看一下伴侶盟語料中含有「同志」的句子：

generate_concordance_df(lgbtfamily_raw_text, '同志')

再看伴侶盟語料中，經過jieba詞性標注後，所有含有「同志」一次句子中的動詞列表：

pd.DataFrame(generate_verb_list(lgbtfamily_raw_text, '同志'))

下福盟中的同志

先看下福盟語料中含有「同志」的句子：

generate_concordance_df(lovefamily_raw_text, '同志')