「同性戀」怎麼了？

這次我們要看的是兩個語料中的「同性戀」參與哪些動作的角色、作為哪些動詞的論元，成為不同敘事中的主角。

以下一樣用 jieba.posseg 所提供的詞性標注功能，針對兩個語料當中，包含「同性戀」的句子進行詞性標註，並篩選出所有的動詞。

import nltk
import re
import json
import jieba
import jieba.posseg as pseg
# from ckip import CkipSegmenter
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib.font_manager import FontProperties
from IPython.display import display
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.decomposition import TruncatedSVD

jieba.set_dictionary('jieba_dict/dict.txt.big')
jieba.load_userdict('jieba_dict/user_dict.txt')
# segmenter = CkipSegmenter()

plt.rcParams['figure.figsize'] = [10, 5]
myfont = FontProperties(fname=r'/usr/local/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/msj.ttf',size=14)
sns.set(font=myfont.get_name())

%matplotlib inline

pd.set_option('display.max_rows', 2000)

LGBTFAMILY_PATH = "raw_content_pro.json"
LOVEFAMILY_PATH = "raw_content_anti.json"

lgbtfamily_raw_text = ""
lovefamily_raw_text = ""

with open(LGBTFAMILY_PATH) as f:
    lgbtfamily_raw_text = json.loads(f.read())

with open(LOVEFAMILY_PATH) as f:
    lovefamily_raw_text = json.loads(f.read())

Building prefix dict from /Users/koukantou/Desktop/Don/ntu_master/107-2-lab-meeting/nano-blog/jieba_dict/dict.txt.big ...
Loading model from cache /var/folders/vp/g4gpl3x91qqdn6_x0t8w4bbc0000gn/T/jieba.u336b009ae6d5715d70945d8db9630e20.cache
Loading model cost 2.965 seconds.
Prefix dict has been built succesfully.

# 前處理函式
def generate_line_sentence(all_text):  
    # 以句號分句
    all_text = re.split('[。，！？!?,]', all_text)
    
    # 去除所有非中文字元
#     all_text = [_get_chinese(s) for s in all_text]
    
    return all_text

# 接收語料(dict)與關鍵詞(string)，回傳語料當中關鍵詞出現的句子中，按照頻率排列的動詞表
def generate_verb_list(raw_text_list, word):
    a = [generate_line_sentence(article['content']) for article in raw_text_list]
    l = [sen for article in a for sen in article if word in sen]
    l = [list(pseg.cut(ll)) for ll in l]
    wordlist = [word for sen in l for (word, pos) in sen if pos.startswith('v')]
    freq_dict = {w:wordlist.count(w) for w in set(wordlist)}
    return sorted(freq_dict.items(), key=lambda kv: -kv[1])

# 接收語料(dict)與關鍵詞(string)，回傳語料中含有該關鍵詞的concordance dataframe
def generate_concordance_df(raw_text_list, word):
    a = [generate_line_sentence(article['content']) for article in raw_text_list]
    l = [sen.split(word) for article in a for sen in article if word in sen]
    df = pd.DataFrame({
    "left_context": [s[0] for s in l],
    "keyword": word,
    "right_context": [s[1] for s in l],
    })
    return df

伴侶盟中的同性戀

先看一下伴侶盟語料中含有「同性戀」的句子：

generate_concordance_df(lgbtfamily_raw_text, '同性戀')

再看伴侶盟語料中，經過jieba詞性標注後，所有含有「同性戀」一次句子中的動詞列表：

pd.DataFrame(generate_verb_list(lgbtfamily_raw_text, '同性戀'))

下福盟中的同性戀

先看下福盟語料中含有「同性戀」的句子：

generate_concordance_df(lovefamily_raw_text, '同性戀')