Разница между страницами «Теория вероятностей и математическая статистика» и «Linguistic Data: Quantitative Analysis and Visualisation: computational linguistics»

Материал из MathINFO
(Различия между страницами)
Перейти к навигации Перейти к поиску
 
 
Строка 1: Строка 1:
−
'''Дорогие студенты!'''
+
* Instructors: Ilya Schurov and Olga Lyashevskaya
  
−
На этой странице будут появляться различные материалы и объявления, связанные с курсом '''«Теория вероятностей и математическая статистика»''', читаемого для студентов 2-го курса школы лингвистики в '''2019/2020''' учебном году.
+
== Materials ==
−
 
+
{| class="wikitable"
−
* Авторы курса: И.В. Щуров, Д.А. Филимонов.
 
−
* Лекции читает: Филимонов Дмитрий Андреевич.
 
−
* Семинары ведет: Филимонов Дмитрий Андреевич.
 
−
 
 
−
<!-- === Таблицы распределений ===
 
−
Таблицы распределений: [http://math-info.hse.ru/f/2017-18/ling-prob/Norm_chi%5E2_Student_Fisher.pdf нормальное, хи-квадрат и Стьюдент]. [http://math-info.hse.ru/f/2017-18/ling-prob/Norm_chi%5E2_Student_Fisher.xlsx Исходники] (открываются не всеми версиями всех программ, созданы в Libre Office 6.0.2.1) -->
 
−
 
 
−
== Материалы ==
 
−
=== Лекции и семинары ===
 
−
{|class='wikitable'
 
−
!дата лекции
 
−
!тема лекции
 
−
!дата семинара
 
−
!задачи к семинару
 
 
|-
 
|-
−
|13.01
+
! Data !! Topics !! Links
−
|Основные понятия теории вероятностей.
 
−
|13.01/14.01
 
−
|[http://math-hse.info/f/2019-20/ling-prob/1_basic_probability.pdf Задачи на основные понятия классической теории вероятностей]
 
 
|-
 
|-
−
|20.01
+
| Jan 18 || Introduction. Quantitative linguistic research and data types. R basics || [https://docs.google.com/presentation/d/1VUIUa3Db5n4dsD_HeA3e-mz55zK8uPrko3yu207pKUk/edit?usp=sharing Intro Slides] [https://github.com/LingData2019/LingData2020/tree/master/seminars/2020-01-18 Lab 01: intro to R]
−
|Формулы полной вероятности и Байеса. Случайная величина. Дискретные случайные величины.
 
−
|20.01/21.01
 
−
|[http://math-hse.info/f/2019-20/ling-prob/2_total_prabability+Bayes.pdf Задачи на формулу полной вероятности и формулу Байеса]
 
 
|-
 
|-
−
|27.01
+
| Jan 25 || Hypothesis testing. Binomial test. R: dataframes, tydyverse || [https://github.com/LingData2019/LingData2020/tree/master/seminars/2020-01-25 Lab 02] [https://datacamp-community-prod.s3.amazonaws.com/e63a8f6b-2aa3-4006-89e0-badc294b179c tidy verse cheat sheet]
−
|Арифметические операции над случайными величинами. Математическое ожидание и дисперсия.
 
−
|27.01/28.01
 
−
|[http://math-hse.info/f/2019-20/ling-prob/3_discrete_random_variable.pdf Дискретная случайная величина]
 
 
|-
 
|-
−
|03.02
+
| Feb 1 || Central limit theorem. Variance. Student's t-test. R: simulating data, boxplots, density plots, binomial test, t-test ||  
−
|Зоопарк дискретных распределений.
+
[https://github.com/LingData2019/LingData2020/tree/master/seminars/2020-02-01 Lab 03: ]
−
|03.02/04.02
+
[https://raw.githubusercontent.com/LingData2019/LingData2020/master/seminars/2020-02-01/Lab3-ttest-binom-matrices.Rmd Rmd] [https://htmlpreview.github.io/?https://github.com/LingData2019/LingData2020/blob/master/seminars/2020-02-01/Lab3-ttest-binom-matrices.html html]
−
|[http://math-hse.info/f/2019-20/ling-prob/4_discrete_rv_zoo.pdf Задачи на различные дискретные случайные величины.]
 
−
|<!-- -
 
−
|10.02
 
−
|Системы дискретных случайных величин.
 
−
|10.02
 
−
|[http://math-hse.info/f/2019-20/ling-prob/5_joint_discrete.pdf  Задачи на системы дискретных случайных величин.]
 
 
|-
 
|-
−
|17.02
+
| Feb 8 || Two-sample T-test. ANOVA. Confidence intervals. Non-parametric tests ||  
−
|Непрерывная случайная величина. Функция распределения. Действия со случайными величинами. Математическое ожидание и дисперсия.
 
−
|17.02
 
−
|[http://math-hse.info/f/2019-20/ling-prob/6_continous_random_variable.pdf  Непрерывная случайная величина.]
 
 
|-
 
|-
−
|02.03
+
| Feb 15 || Tests for categorial data. Chi-squared test. Fisher exact test. Effect size. ||
−
|Зоопарк непрерывных распределений: Показательное, Нормальное и Парето.
 
−
|02.03
 
−
|[http://math-hse.info/f/2019-20/ling-prob/7_continous_rv_ED.pdf Непрерывная случайная величина. Математическое ожидание и дисперсия.]
 
 
|-
 
|-
−
|16.03
+
| Feb 22 || Correlations. Linear regression || [https://lindeloev.github.io/tests-as-linear/linear_tests_cheat_sheet.pdf Common statistical tests & linear models ]
−
|Неравенство Чебышёва. Закон больших чисел, центральная предельная теорема и теорема Муавра-Лапласа.  
 
−
|16.03
 
−
|[http://math-hse.info/f/2019-20/ling-prob/8_normal+M-L.pdf Нормальное распределение, действия с непрерывными случайными величинами и теорема Муавра-Лапласа.]
 
 
|-
 
|-
−
|23.03
+
| Feb 29 || Multivariate linear regression. Dummy variables ||  
−
|'''Контрольная'''
 
−
|
 
−
|
 
 
|-
 
|-
−
|06.04
+
| || Dimensionality reduction. PCA. MDS. t-SNE ||
−
|Математическая статистика: выборка, ее характеристики.
 
−
|06.04
 
−
|[http://math-hse.info/f/2019-20/ling-prob/9_sample-quantile-mean.pdf  Выборки. Квантили и среднее.]
 
 
|-
 
|-
−
|13.04
+
| || CA, MCA. Clusterization ||  
−
|Точечные оценки и их свойства.  
 
−
|13.04
 
−
|[http://math-hse.info/f/2019-20/ling-prob/10_point_estimation.pdf Точечные оценки.]
 
 
|-
 
|-
−
|20.04
+
| || Logistic regression. Model selection ||  
−
|Интервальные оценки. Распределение Стьюдента.
 
−
|20.04
 
−
|[http://math-hse.info/f/2019-20/ling-prob/11_interval_estimation.pdf Интервальные оценки.]
 
 
|-
 
|-
−
|27.04
+
| || Fixed and random effects. Linear mixed-effects models ||  
−
|Статистические критерии. Z-тест и тест Стьюдента.
 
−
|27.04
 
−
|[http://math-hse.info/f/2019-20/ling-prob/12_z-and_t-test.pdf Z-критерий и критерий Стьюдента.] '''Показ работ.'''
 
 
|-
 
|-
−
|11.05
+
| || Bootstrap. Decision trees. Decision forests ||  
−
|Критерии хи-квадрат. Таблицы сопряженности. Точный критерий Фишера.
 
−
|11.05
 
−
|[http://math-hse.info/f/2019-20/ling-prob/13_chi^2-test.pdf Критерии хи-квадрат.]
 
 
|-
 
|-
−
|18.05
+
| || Bayesian statistics ||  
−
|Дисперсионный анализ и поправки.
 
−
|18.05
 
−
|[http://math-hse.info/f/2019-20/ling-prob/14_ANOVA+power.pdf ANOVA и мощность z-критерия.]
 
 
|-
 
|-
−
|25.05
+
| || Bayesian statistics II ||  
−
|Различные типы шкал. Мера ассоциации величин, корреляция.
 
−
|25.05
 
−
|Продолжение работы с предыдущим листочком.
 
 
|-
 
|-
−
|01.06
+
|}
−
|Регрессия, дайджест курса.
+
 
−
|01.06
+
== Software ==
−
|[http://math-hse.info/f/2019-20/ling-prob/15_correlation+regression.pdf Корреляция и регрессия.]
+
During this course we will use R as a programming language and RStudio as a GUI.
−
|-
+
 
−
|
+
How to install R and RStudio?
−
|
+
 
−
|08.06
+
1. Download [https://cran.r-project.org/ R] (you can choose another mirror here if you wish) and install it on your computer. Make sure you did it before installing RStudio.
−
|Продолжение работы с предыдущим листочком.
+
 
−
|-
+
2. Download [https://rstudio.com/products/rstudio/ RStudio] (you need RStudio Desktop Open Source License) and install it on your computer. It is recommended to create a shortcut for RStudio during installation.
−
|??.06
+
 
−
|'''Экзамен'''
+
It is possible avoid installing anything on your PC, using [https://rstudio.cloud rstudio.cloud] (an online version of RStudio).
−
|??.06
+
 
−
|'''Показ работ'''
+
For successful submission of assignments you should be able to create and save R code files (.R) and RMarkdown files (.Rmd).
−
|-->}
+
 
 +
 
 +
== Homeworks ==
 +
* Homework 1 (deadline: February 16, 23:59), Chapters 1, 2, 3, and 5 of the [https://www.datacamp.com/courses/free-introduction-to-r DataCamp] course "Introduction to R". Please fill in this [https://docs.google.com/forms/d/e/1FAIpQLSdjgKBM5JSo6D6ajhrWWfFG1ktcKgDfbdK_jQ_ZbW9GwNLzpQ/viewform form].
 +
* Homework 2 (deadline: February 23, 23:59), Chapters 4 and 6 of the [https://www.datacamp.com/courses/free-introduction-to-r DataCamp] course "Introduction to R". 
 +
After completing the course please provide either the [https://support.datacamp.com/hc/en-us/articles/360001548814-How-can-I-share-my-certificate-Statement-of-Accomplishment- Statement of Accomplishment] or a screenshot of your learning progress via [link TBA]. 
 +
Deadlines for Homework 1 and 2 are cancelled due to unavailability of the free version of the datacamp online course. Stay tuned!
 +
* Homework 3 (deadline: February 9, 12:00), Hypothesis testing, binomial test, t-test. [https://github.com/LingData2019/LingData2020/blob/master/hw/hw-pdf/LingData-HW3-comp.pdf HW3 pdf] [https://htmlpreview.github.io/?https://github.com/LingData2019/LingData2020/blob/master/hw/LingData-HW3-comp.html html] [https://github.com/LingData2019/LingData2020/blob/master/hw/LingData-HW3-comp.Rmd Rmd template]
 +
 
 +
== Final project ==
 +
* Projects description [https://github.com/LingData2019/LingData2020/blob/master/projects.pdf link] 
 +
* Projects pre-registration: link to submit your file TBA 
 +
* Final versions of project papers: link to sumbit your files TBA 
 +
 
 +
 
 +
== References ==
 +
* Gries, Stefan (2013). Statistics for Linguistics with R : A Practical Introduction (Vol. 2nd revised edition). Berlin: De Gruyter Mouton. [http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=604318 HSE library link]
 +
* Levshina, Natalia (2015). How to Do Linguistics with R : Data Exploration and Statistical Analysis. Amsterdam: John Benjamins Publishing Company. [http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=nlebk&AN=1093048 HSE library link]
 +
* Baayen, Harald (2008). Analyzing Linguistic Data: A practical introduction to statistics. Cambridge UP. [http://www.sfs.uni-tuebingen.de/~hbaayen/publications/baayenCUPstats.pdf pdf]
 +
 
 +
* Gries, Stefan (2017). Quantitative Corpus Linguistics with R : A Practical Introduction (Vol. Second edition). Milton Park, Abingdon, Oxon: Routledge. eBook
 +
* Empirical Bayes
 +
* Harney, H. L. (2016). Bayesian Inference : Data Evaluation and Decisions (Vol. 2nd ed). Springer. eBook 
 +
* McElreath, R. (2016). Statistical Rethinking : A Bayesian Course with Examples in R and Stan. eBook
 +
* ggplot2
 +
* Hadley, W. (2016). Ggplot2 : Elegant Graphics for Data Analysis. Springer. eBook
 +
* R markdown [https://rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf Rmd Cheat Sheet
  
−
== Литература ==
+
== Course Info ==
−
С базовой теорией вероятностей можно знакомиться по учебнику [1]. На более глубоком уровне существует много учебников по вероятности и статистике. Например, можно читать [2]. Из учебников, доступных в электронном виде, отметим очень неплохую книгу [3] (на английском), см. главу 2.
 
  
−
# Тюрин Ю. Н., Макаров А. А., Симонова Г. И. [http://biblio.mccme.ru/node/2179 Теория вероятностей. Учебник для экономических и гуманитарных специальностей.] М.: МЦНМО, 2009.
+
This page contains the materials of the course "Linguistic Data: Quantitative Analysis and Visualisation", taught at the HSE Master's program "Computational Linguistics" in 2019-2020 academic year. Modules: 3-4.
−
# Кремер. Н. Ш. [http://www.unity-dana.ru/index.php?page=shop.product_details&flypage=shop.flypage&product_id=924&category_id=23&manufacturer_id=0&option=com_virtuemart&Itemid=26 Теория вероятностей и математическая статистика.] М.: Юнити-Дана, 2010.
 
−
# David M Diez, Christopher D Barr, Mine Cetinkaya-Rundel. [http://www.openintro.org/stat/textbook.php OpenIntro Statistics] Second Edition.
 
−
# David H. Freedman, Robert Pisani, Roger Purves. Statistics.
 

Версия 09:54, 8 февраля 2020

  • Instructors: Ilya Schurov and Olga Lyashevskaya

Materials

Data Topics Links
Jan 18 Introduction. Quantitative linguistic research and data types. R basics Intro Slides Lab 01: intro to R
Jan 25 Hypothesis testing. Binomial test. R: dataframes, tydyverse Lab 02 tidy verse cheat sheet
Feb 1 Central limit theorem. Variance. Student's t-test. R: simulating data, boxplots, density plots, binomial test, t-test

Lab 03: Rmd html

Feb 8 Two-sample T-test. ANOVA. Confidence intervals. Non-parametric tests
Feb 15 Tests for categorial data. Chi-squared test. Fisher exact test. Effect size.
Feb 22 Correlations. Linear regression Common statistical tests & linear models
Feb 29 Multivariate linear regression. Dummy variables
Dimensionality reduction. PCA. MDS. t-SNE
CA, MCA. Clusterization
Logistic regression. Model selection
Fixed and random effects. Linear mixed-effects models
Bootstrap. Decision trees. Decision forests
Bayesian statistics
Bayesian statistics II

Software

During this course we will use R as a programming language and RStudio as a GUI.

How to install R and RStudio?

1. Download R (you can choose another mirror here if you wish) and install it on your computer. Make sure you did it before installing RStudio.

2. Download RStudio (you need RStudio Desktop Open Source License) and install it on your computer. It is recommended to create a shortcut for RStudio during installation.

It is possible avoid installing anything on your PC, using rstudio.cloud (an online version of RStudio).

For successful submission of assignments you should be able to create and save R code files (.R) and RMarkdown files (.Rmd).


Homeworks

  • Homework 1 (deadline: February 16, 23:59), Chapters 1, 2, 3, and 5 of the DataCamp course "Introduction to R". Please fill in this form.
  • Homework 2 (deadline: February 23, 23:59), Chapters 4 and 6 of the DataCamp course "Introduction to R".

After completing the course please provide either the Statement of Accomplishment or a screenshot of your learning progress via [link TBA]. Deadlines for Homework 1 and 2 are cancelled due to unavailability of the free version of the datacamp online course. Stay tuned!

Final project

  • Projects description link
  • Projects pre-registration: link to submit your file TBA
  • Final versions of project papers: link to sumbit your files TBA


References

  • Gries, Stefan (2013). Statistics for Linguistics with R : A Practical Introduction (Vol. 2nd revised edition). Berlin: De Gruyter Mouton. HSE library link
  • Levshina, Natalia (2015). How to Do Linguistics with R : Data Exploration and Statistical Analysis. Amsterdam: John Benjamins Publishing Company. HSE library link
  • Baayen, Harald (2008). Analyzing Linguistic Data: A practical introduction to statistics. Cambridge UP. pdf
  • Gries, Stefan (2017). Quantitative Corpus Linguistics with R : A Practical Introduction (Vol. Second edition). Milton Park, Abingdon, Oxon: Routledge. eBook
  • Empirical Bayes
  • Harney, H. L. (2016). Bayesian Inference : Data Evaluation and Decisions (Vol. 2nd ed). Springer. eBook
  • McElreath, R. (2016). Statistical Rethinking : A Bayesian Course with Examples in R and Stan. eBook
  • ggplot2
  • Hadley, W. (2016). Ggplot2 : Elegant Graphics for Data Analysis. Springer. eBook
  • R markdown [https://rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf Rmd Cheat Sheet

Course Info

This page contains the materials of the course "Linguistic Data: Quantitative Analysis and Visualisation", taught at the HSE Master's program "Computational Linguistics" in 2019-2020 academic year. Modules: 3-4.

Источник — http://math-info.hse.ru/wiki2019-20/index.php?title=Теория_вероятностей_и_математическая_статистика&oldid=82