Analysis Of Dictum-Modus Contents Of An Utterance By Methods Of Artificial Intelligence

Abstract

The article discusses the problems of optimization of methods of automatic analysis of texts in natural language. It is emphasized that modern methods of computer text analysis, including the method of relational-situational analysis (RSA) used by the authors, are mainly based on modeling the proposition semantics of sentences, or dictum, while the information about the subject of speech, or modus, due to the specificity of the means and methods of expression, remains largely inaccessible to automatic analysis. The authors believe, that the RSA method, focused on the perception of multilevel linguistic features of an utterance, may well be used to identify modus characteristics of the text. The RSA model is a heterogeneous semantic network, for which linguistic templates are used. Application of the psychological principle of objectivity in the method of linguistic templates allows one to move from a statistical description to a meaningful analysis of the mental actions of the subject and its motives realized with the help of the text. Further development of automatic text analysis systems, according to the authors, requires equipping the linguistic analyzer with templates that include speech markers of the modus. Since the dictionary of predicate words is the core of the relational-situational analyzer, the problem of studying the modus potential of verbs and other predicate words is of particular importance. It is emphasized that the appeal to the subjective components of the meaning of text units dictates going beyond the limits of the language system into speech system.

Keywords: Dictumlinguistic analyzermodusrelational-situational analysisspeech systematicity

Introduction

In works on artificial intelligence (AI), the acquisition of knowledge involves the identification of not only linguistic, but also logical (in the broadest sense of the word) relationships in language objects (Osipov, 2011; Osipov et al., 2018). It is not surprising, therefore, that methods of computer semantic analysis are based on logical and syntactic concepts of the meaning of a statement. The most significant influence on the development of these methods was made by the studies of the Cambridge Linguistic Circle, the transformational grammar of N. Chomsky (as cited in Osipov, 2011), the formal semantics of R. Montague(as cited in Osipov, 2011), the case grammar of C. Fillmore (Osipov, 2011), the “Sense-Text” model by I. A. Melchuk et al. (as cited in Osipov, 2011). Especially fruitful was the modeling of the propositional (dictum) level of meaning of the sentence – reflection of the situation structure (relational structure) by the proposal. This modeling involves attributing the participants of the situation of semantic-syntactic roles, or universal meanings of the deep cases – such as agentive, instrumentalis, dative, factitive, locative, objective, etc. Roles are determined by the lexical meaning of the predicate word, most often a verb in which the structure of the reflected situation is fixed, as it were (Boldyrev, 2019).

In works on automatic acquisition of knowledge and solution of other applied problems, a major achievement was the concept of relational-situational analysis of text (Kuznetsova et al., 2019; Osipov, 2011; Osipov et al., 2018). In the field of linguistics, it relies on the functional syntax of the Russian language (Zolotova, 2010), in the field of AI, on the theory of heterogeneous semantic networks (Osipov, 2011).

Since the semantics of a sentence is a whole that cannot be reduced to the sum of the values of its parts, the relational-situational analysis of a sentence is aimed at revealing not only its syntaxemes and their meanings, but also the semantic relations between different meanings. For this, lists of 1) meanings of syntaxemes (semantic-syntactic roles) with a list of means of their expression, presented in Russian, 2) types of binary relationships on sets of these values are used. Examples of the first: Instrumentative is an instrument of action (That which is written with a pen cannot be cut out with an axe (The pen is mightier than the sword). Destinative – the appointment of an action or object (to advocate for animals; to go for treatment). Examples of the second: INS: (subject, instrumentative), DES: (object, destinative), etc. (Osipov, 2011). As a result of the definition of syntaxemes and a set of relations in the text, the description of the structures of semantic information in the form of a relational-situational model of the text is carried out. Formally, these structures are heterogeneous semantic networks (Osipov, 2011).

It is important to add to the above that the relational-situational analysis is aimed at conventional information expressed by a linguistic sign (in a word, a sentence model). This analysis does not extend to information associated with a sign in the minds of communicants.

So, modern methods of computer analysis of the meaning of sentences are focused mainly on the modeling of dictum. Meanwhile, the proposal has not only a dictum, but also a modus plan. Recently there have been studies aimed at solving various applied problems associated with the analysis of the modal content of speech. The directions of analysis of sentiment of utterances (the attitude of the author of the text to the mentioned objects) and manifestations of verbal aggression (Chudova & Panov, 2016; Kuznetsova, 2017) are developing very actively. Since a large number of marked-up texts devoted to various topics have already been accumulated during the study of sentiment, the main attention is paid to the problem of training complex models (multilayer neural networks) with a teacher. So, in the work of Rosenthal et al. (2019), teacher training with multilayer neural networks containing recurrent layers is used, which allows one to take into account the context of statements. A similar approach is used by Do et al. (2019), Zhou et al. (2019).

The second actively developing area is the analysis of the emotional “state of charge” of an utterance. In the work of Vybornova et al. (2011) a set of linguistic features (speech markers) that can be used to assess the degree of emotional tension of the author is presented. Multilevel attributes are taken into account, including the presence of certain lexical items, syntactic constructions, predicate words with a given set of arguments in the text. A similar approach was also used to assess the level of aggression expressed in the text (Devyatkin et al., 2014).

In all these studies, certain problems related to the assessment of the modus content of speech are solved, however, to create a full-fledged analyzer, it is necessary not to simply identify markers of social attitudes, values, emotional states and the degree of saturation of texts and their fragments with these markers, but to proceed to a systematic study of diverse modus characteristics of an sentence-utterance.

Problem Statement

Let us briefly dwell on the question of the main manifestations of the modus and a possible way of formalizing them, relying on the relational-situational method of analysis of utterances, supplemented by the use of templates. As known, dictum is information about the reality behind the sentence, and modus is the representation of the subject of speech in the utterance. A sentence-utterance can take place only if information about the world in this utterance is combined with subjective information. The latter usually includes: the purposefulness of the utterance, its truthfulness, the speaker’s assessment of the relationship of the communicant to reality, various expressive shades of the utterance (Shmeleva, 1994).

The degree of representation in the semantic organization of the utterance of dictum and modus meanings depends on its function in conversation. According to the classical scheme of Buhler (2000), the representative function of a linguistic sign is manifested in relation to “objects and state of affairs”, expressive – to the sender (speaker), appellative – to the receiver (addressee). Language is best prepared and equipped to implement a representative function (Buhler, 2000). And this is no coincidence. According to Vygotsky (2019), it is the function of designating objects of reality that is specific to the human language and it is it that provides the appearance of such a form of mental reflection as consciousness, which allows the subject to create an image of reality that is separate from the subject’s relationship to it (Leontiev, 2020).

The most illustrative example of implementation of the representative function of a linguistic sign is the scientific language, of the expressive function – the lyrics expressing an author’s impressions, their volitional impulses, emotions, and of the appellative function – the command language that controls the actions of an addressee (Buhler, 2000). It is not surprising, therefore, that the relational-situational method is most productive in the study of scientific texts, as well as texts of informational media genres.

When solving text processing problems, the linguistic analyzer takes into account the formal and grammatical indicators of the categorical meanings of verbs and names (Osipov, 2011), aggregated by the relational-situational model. To work with the model, templates that specify a lot of heterogeneous semantic networks that have a certain common property, are used, which allows one to take into account multilevel linguistic features necessary for solving applied problems.

A psychologist, a linguist and a programmer participate in the preparation of the template. First, the psychologist reveals the essential features of the simulated mental processes (cognitive-affective actions, operations), which, refracting the cognitive object and embodying in linguistic forms, form the conventional meaning of the utterance. Then, the linguist, if possible, fully and accurately describes the means of expression not only of this conventional meaning, but of all information that is arbitrarily and involuntarily transmitted by the speaker to the listener. In other words, a linguist fixes markers of information associated with an utterance in the minds of the communicants. The subject of the analysis is in this case the use of multilevel language units as a means of embodiment of mental (psychological) processes. At the final stage, the programmer formalizes the linguistic description. Thus, with the help of templates, the bridge is thrown from the categorical-grammatical meaning of the utterance to its cognitive-affective content.

The theoretical basis for the work of an AI specialist with linguistic templates is the idea of developmental psychology about the existence of various cultural standards of perception and thinking, as well as the concept of the objectivity of mental reflection (Leontiev, 1976). According to these notions, the information supplied to the inputs of the analyzer systems is divided up in accordance with the practice of using objects in society (for animals – in accordance with the practice prescribed by instinctive behavior programs). As a result, the sensory-perceptual system processes not the isolated from the environment signals, but the information about objects. The subjective nature of perception, determined by the organizing action of the standards, allows one to recognize objects and events that are important for the subject, their own and others’ actions, the intentions of the subjects of interaction: instead of tracking a set of parameters measured by neurophysiological detectors, the psyche subject operates with rules for identifying nonrandom structures, structures that reflect something significant in the subject’s life. Accordingly, the application of the principle of objectivity in the method of linguistic templates allows one to move from statistical comparisons to a meaningful analysis of the subject's actions (including mental ones) and the subject’s motives, including unconscious ones.

Let us explain what was said with the help of illustrations. The first example will be the statements of a scientific text, which, as was noted, express mainly dictum meanings. The second example is statements that are the reaction of the communicants to a frustrating situation. Their content is dominated by modus meanings.

1) The author of a scientific article fixes the established pattern of a cause-effect type: An increase in temperature causes an increase in the duration of the phase. A decrease in solar insolation hours causes a delay in ripening.

The dictum categorical-grammatical semantics of such statements are close to the typical content of the cognitive actions embodied in them. Therefore, the relational-situational method is an effective means of automatic recognition of the latter. The use of templates allows one to improve the result, since they contain precisely those linguistic features that are characteristic for the performance of this cognitive-speech action, in contrast to other actions. Reducing the set of formalized and aggregated features provided by the linguistic analyzer device to a set of differential features improves the recognition of cognitive processes and significantly reduces the size of the prepared test sample.

2) The subject of speech reacts to a situation of obstacles or accusations (according to the test of S. Rosenzweig): a) Change your seat if you are not comfortable with something; Drive already; Throw it away in that case; b) The child needs to be looked after; You need to slow down; It is worthwhile to warn one about it whenever possible; c) Well, why would I need a torn newspaper ?; And why should I wait another three hours?; And how does this relate to me?

The dictum sense of all these statements is different (and not very significant), and the modus sense – the most important sense for the communicants – coincides in each group. In the examples of the group (a), it is the confrontational urge of the interlocutor to take certain actions. The main formal means of expressing this modus meaning is the form of the imperative mood of the verb. The utterances of the group (b) again have a confrontational modus semantics of edification, teaching, expressed primarily by an impersonal-predicative word with an adjoining infinitive. Statements presented in the group (c) are a confrontational objection in the form of a rhetorical question, i.e. expressive form of non-interrogative statement expressing indignation, categorical disagreement with the position of the interlocutor. The main indicators of this modus content are pronoun adverbs (why, what for, how. etc) and intonation, the role of which in the embodiment of modus meanings is always significant.

We emphasize that the relational-situational analysis of utterances, if it is not supplemented by the identification of linguistic features of the template in them, allows one to distinguish between psychological types of reactions to a frustrating situation only if there are certain semantic roles. However, this is clearly not enough for the automatic recognition of modus meanings that reveal certain cognitive-emotive actions and states of communicants.

The illustrations above emphasize the importance of creating a software tool that would expand the capabilities of the analyzer, focused on relational-situational modeling of sentences, and would allow for relatively complete fixation of both dictum and modus meanings. From a linguistic point of view, the solution to this problem involves the inclusion in the analysis of not only the semantic-syntactic relationships between syntaxemes as constituents of a sentence, but also its well-known modifications, which include the use of forms of surreal moods in secondary functions; in the so-called regular realizations (incompleteness of the implementation of the predicative basis, the introduction of connectives and connective formations, semi-auxiliary verbs, etc.); in filling syntactic positions with the words of certain lexical-semantic classes; in the features of the distribution of the proposal; in a grammatically significant word order.

Addressing these aspects of sentence modeling involves shifting attention from a functionally appropriate device of the language system to the speech system (Kozhina, 2019), which always develops during the work of the language mechanism (which determines the importance of formalizing actant-predicate relations), but does not boil down to the implementation of the language system, but involves a speech increase: selection, repetition, combination, modification of language tools.

Research Questions

The core of the relational-situational analyzer is a dictionary of predicate words, based on the semantic-grammatical classification of verbs (Osipov, 2011). Therefore, the further development of the analyzer, which involves identification of the modus meanings of the sentence, requires the study of the modus potential of verbs and predicates.

As known, the semantic twist in the syntax manifested itself, firstly, in the introduction into the syntax of the communicative concept of “utterance” and the semantic category “situation” (also denoted by the terms “state of affairs”, “event”, “proposition”, “dictum”, etc), and secondly, the involvement in the orbit (and often the focus) of the analysis of the subjective components of the content of the utterance – self-referent "ego-meanings". As a special aspect, the nominative aspect of the sentence was highlighted. The structural and semantic center of the proposition, its “plenipotentiary,” is the verb (or other predicate word). And if the noun in itself (as a component of the lexical system of the language) does not carry enough information to make it possible to get an idea of the proposition in which it is able to play one or another semantic role, then the verb, predicative adverb or adjective is already outside the speech act, it integrally represents (in the speaker’s mind) the entire proposition of the potential utterance. And only during the speech act, the proposition, being included in the communicative coordinate system “I am here, now,” is endowed with ego-senses and receives the status of a full-fledged communicative unit – a complex sign with “external” (dictum) and “internal” (modus) reference and a definite theme-rheumatic division. The external reference (correlation with the event/state of affairs) is possessed by the dictum part of the meaning of the utterance, the modus and performative framework are the internal reference.

It is important to distinguish between the modus itself (“ego-meanings”) and modal (in the broad sense, including emotive and evaluative) predicates in the dictum use. Compare: I think that Ivan will not come – Peter thinks that Ivan will not come. Moreover, even the use of the emotive verb of first person singular is not a sufficient basis for the conclusion that there is a modus ( I suffer is similar in content to the expressions He suffers, You suffer ). Note that one of the signs of modus predicates (the modus potential of a language, if you use the term used by T. V. Shmeleva) is their inclusion in the category of "meta-predicates" – explanatory support words that subordinate semiotic units of the lower (primary) level, or dictum. If a non-explanatory emotive predicate is semantically self-sufficient and can completely do without a sign with an external referent, then this is impossible for a modus: I think (I am afraid, I hope) without a subordinate clause or a positive anaphora (that) is possible only in a dialogical context.

• One of the important features of the modus, which do little to automate the procedures of semantic analysis, is a tendency towards an external expression and a high degree of homogeneity of its verbal manifestations (Shmeleva, 2019). Therefore, the most important task is to find ways to resolve the lexical homonymy of predicate words that can be explicit indicators of modus and modal meanings.

The core of the relational-situational analyzer is a dictionary of predicate words, based on the semantic-grammatical classification of verbs (Osipov, 2011). Therefore, the further development of the analyzer, which involves identification of the modus meanings of the sentence, requires the study of the modus potential of verbs and predicates.

There is no doubt that, due to the features of the modus noted above, creation of algorithms for reliable identification of various modus senses is a very difficult task. At this stage of the research, we consider it important to determine which segment or suprasegmental indicators can, in principle, be regarded as markers of the mental states of the speaking subject, as well as states attributed to the addressee of speech or to a third party (in the second case, these indicators are apparently always explicit and therefore, it suffices to list the predicates of the corresponding semantic categories and find the optimal means of automatically resolving homonymy). But in the case of the modus itself (the mental state of the speaker), explicit markers of usually implicit performative-modus frames are quite common, i.e. reflexive verbs of first person singular of present tense (in semantic reconstructions, these verbs are underlined). Compare the above examples of “frustration reactions”: Change your seat if you are not comfortable with something = [{Since you don’t want / to be next to me at the moment, which I don’t approve} I advise ...]; The child needs to be looked after = [{I negatively assess the fact that you are not looking after the child here, therefore, as a edification] I tell you ...]; Well, why would I need a torn newspaper? = [{I regard your actions as aimless, therefore} I say ...] (in curly brackets there are implicit meanings that are "read" with high probability, but only in a certain context and in a certain communication situation).

Purpose of the Study

The goal of the study, determined by the statement of the problem, is to develop theoretical foundations for improving the relational-situational analyzer, designed to fix both dictum and modus meanings.

Research Methods

Various approaches can be used to create an analyzer of the modus content of an utterance. Let us briefly describe them. Unsupervised learning involves refusing to use the learning sample. The correctness of the solutions obtained this way is ensured by the features of model training, for example, the use of specific loss functions, regularizers. An example of application of this approach is the work (Drozdov et al., 2019), in which a tool for constructing syntactic coverage trees is trained this way.

Unsupervised learning, as a rule, is used when solving problems with a small number of classes, the allocation of which is quite obvious. In other cases, it is necessary to use active learning. The process of active learning is iterative. It starts on a small marked-up corpus, on which the analyzer is built and tested, then the system requests additional marked-up examples for classes that include the largest number of errors (Shelmanov et al., 2019).

Another approach is transfer learning. The training consists in presetting the analyzer with a teacher on the labeled corpora designed to solve a specific problem, and then in using this analyzer (possibly with additional training) to solve another, similar problem.

For reliable construction of analyzers in the conditions of a limited volume of labeled data, it is necessary to reduce the complexity of the models used. The obvious solution is to reduce the dimension of the attribute space, which uses the methods of generating a compact vector representation of vocabulary, text fragments, as well as language models (Peters et al., 2018). In all these cases, training is carried out unsupervised in large unallocated corpora. It allows one to identify high-level distinctive features that reflect some general patterns of text organization; however, these signs are not interpretable.

An alternative solution, consistent with the semantic-syntactic concept of modus, is to form a limited set of features using knowledge on how modus is expressed in an utterance. This approach allows one to use relatively simple models that are reliably trained on small data sets.

Based on the results of this brief review, it can be stated that there are currently no universal formal models for representing the modal content of texts; existing works are focused on solving particular problems.

Relational-situational analysis is a way of constructing models that reflect the dictum content of utterances, however, these models, as aggregating multilevel linguistic features of an utterance, can be used to identify not only dictum, but also modus characteristics of the text.

The training of a promising analyzer of dictum-modus speech content involves the use of marked-up corpora of texts of a limited size. One of the approaches to solving the problem can be the use of templates to form a high-level feature description of a small dimension based on the results of relational-situational analysis.

Findings

Methods of computer semantic analysis, based on the logical and semantic concepts of the meaning of an utterance, are focused on modeling mainly the dictum – the reflection of the situation structure by the sentence. The task of creating a software tool that would allow one to relatively fully fix both dictum and modus meanings becomes relevant.

In linguistic terms, the solution to this problem involves the inclusion in the study not only of the semantic-syntactic relationships between the syntaxemes of the sentence, but also its various modifications (the use of surreal moods in secondary functions, filling in the syntactic positions with words of certain semantic groups, distribution features, etc). The emphasis in modeling is shifted from linguistic consistency to verbal consistency.

The primary task of research is to identify speech markers of subjective meanings, primarily predicates that can express modus, and formalized speech attributes of communicative units, the use of which is due to the discursive usage and without which reliable identification of the modal value of predicates is unattainable.

Conclusion

Among the various approaches to creating a linguistic analyzer, the task most closely corresponds to the use of templates to form a high-level attribute description of a small dimension based on the results of a relational-situational analysis.

Acknowledgments

The authors are grateful to the Russian Foundation for Basic Research (scientific projects No. 17-29-07049 ofi_m and 17-29-02247 ofi_m.) for the financial support.

References

1. Boldyrev, N. N. (2019). Yazyk i sistema znanij. Kognitivnaya teoriya yazyka [Language and the system of knowledge. A cognitive theory of language]. YASK.
2. Buhler, K. (2000). Teoriya yazyka [The theory of language]. Progress.
3. Chudova, N. V., & Panov, A. I. (2016). Izvlechenie prichinno-sledstvennyh otnoshenij iz dannyh psihologicheskogo issledovaniya na materiale izucheniya agressivnosti [Causal inferences in psychological data in the case of aggression]. Iskusstvennyj intellekt i prinyatie reshenij [Artificial intelligence and decision making], 4, 38-46.
4. Devyatkin, D. A., Kuznetsova, Yu. M., Chudova, N. V., & Shvec, A. V. (2014). Intellektualnyj analiz proyavlenij verbalnoj agressivnosti v tekstah setevyh soobshchestv [An intellectual analysis for the social web: recognizing verbal aggression]. Iskusstvennyj intellekt i prinyatie reshenij [Artificial intelligence and decision making], 2, 27-41.
5. Do, H. H., Prasad, P. W. S., Maag, A., & Alsadoon, A. (2019). Deep learning for Aspect-Based Sentiment Analysis: a comparative Review. Expert Systems with Applications Journal, 118, 272-299.
6. Drozdov, A., Verga, P., Yadav, M., Iyyer, M., & McCallum, A. (2019). Unsupervised latent tree induction with deep inside-outside recursive autoencoders. ArXiv, 1904.02142v2. https://arxiv.org/pdf/1904.02142.pdf
7. Kozhina, M. N. (2019). Rechevaya sistemnost funkcionalnogo stilya [Functional style speech system]. In M. N. Kozhina (Ed.), Stilisticheskij enciklopedicheskij slovar russkogo yazyka [Stylystic Encyclopedic Dictionary of the Russian Language] (pp. 347-350). Flinta, Nauka.
8. Kuznetsova, Yu. M. (2017). O prediktorah pokazatelej agressivnosti[Predictors of aggressiveness indicators]. Iskusstvennyj intellekt i prinyatie reshenij [Artificial intelligence and decision making], 2, 74-77.
9. Kuznetsova, Yu. M., Smirnov, I. V., Stankevich, M. A., & Chudova, N. V. (2019). Sozdanie instrumenta avtomaticheskogo analiza teksta v interesah socio-gumanitarnyh issledovanij. Chast 2. Mashina RSA i opyt ee ispolzovaniya [Creating a text analysis tool for socio-humanitarian research. part 2. RSA machine and the experience of using it]. Iskusstvennyj intellekt i prinyatie reshenij [Artificial intelligence and decision making], 3, 40-51. DOI: 10.14357/20718594190305
10. Leontiev, A. N. (1976). O putyah issledovaniya vospriyatiya (vstupitelnaya statjya) [On the rituals of the study of perception (introductory article)]. In A. N. Leontiev (Ed.), Vospriyatie i deyatelnost [Perception and activity] (pp. 3-27). MGU.
11. Leontiev, A. N. (2020). Problemy razvitiya psihiki [Problems of development of the psyche]. Smysl.
12. Osipov, G. S. (2011). Metody iskusstvennogo intellekta [Artificial intelligence methods]. Fizmatlit.
13. Osipov, G. S., Chudova, N. V., Panov, A. I., & Kuznecova, Yu. M. (2018). Znakovaya kartina mira subjekta povedeniya [Symbolic picture of the world of the subject behaviour]. Fizmatlit.
14. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. ArXiv, 1802.05365. https://arxiv.org/pdf/1802.05365.pdf
15. Rosenthal, S., Farra, N., & Nakov, P. (2019). Sentiment analysis in Twitter. In S. Bethard, M. Carpuat, M. Apidianaki, S. M. Mohammad, D. Cer, & D. Jurgens (Eds.), Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) (pp. 502-518). Vancouver: Association for Computational Linguistics. https: https://arxiv.org/pdf/1912.00741.pdf
16. Shelmanov, A., Liventsev, V., Kireev, D., & Khromov, N. (2019). Active Learning with Deep Pre-trained Models for Sequence Tagging of Clinical and Biomedical Texts. In I. Yoo, J. Be, & X. Hu (Eds.), 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 482-489).
17. Shmeleva, T. V. (1994). Semanticheskij sintaksis [The semantic syntax]. Krasnoyarsk State University.
18. Shmeleva, T. V. (2019). Plan vyrazheniya modalnosti: problema polnoty opisaniya [The expression of modality: the problem of the comprehensive description]. Trudy Instituta russkogo yazyka im. V.V. Vinogradova [Proceedings of the V.V. Vinogradov Russian Language Institute], 20, 329-336.
19. Vybornova, O., Smirnov, I., Sochenkov, I., Kiselyov, A., Tikhomirov, I., & Osipov, G. (2011). Social tension detection and intention recognition using natural language semantic analysis: On the material of Russian-speaking social networks and web forums. In N. Memon, & D. Zeng (Eds.), 2011 European Intelligence and Security Informatics Conference (pp. 277-281). IEEE. https://doi.org/
20. Vygotsky, L. S. (2019). Myshlenie i rech. Psihologicheskie issledovaniya [Thinking and speaking: Psychological studies]. Nacionalnoe obrazovanie.
21. Zhou, J., Lu, Y., Dai, H.-N., Wang, H., & Xiao, A. H. (2019). Sentiment analysis of Chinese microblog based on stacked bidirectional LSTM. IEEE Access, 7, 38856-38866. https://www.henrylab.net/wp-content/uploads/2019/04/08667413.pdf
22. Zolotova, G. A. (2010). Kommunikativnye aspekty russkogo sintaksisa [Communicative aspects of Russian syntax]. URSS.