How can i calculate very large distance matrix? (2024)

4 views (last 30 days)

Show older comments

Ashutosh Shukla on 24 Dec 2020

  • Link

    Direct link to this question

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix

  • Link

    Direct link to this question

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix

Edited: Ashutosh Shukla on 2 Jan 2021

Open in MATLAB Online

I am working on 2D simulation of particles moving in a plane. The particles interact with other particles in some interaction radius, r. Thus i have to calculate interparticle distances, which gets messy as number of particles gets large. (I need large number of particles for better statistics and less boundary effects.)

I am currently doing this:

N=1024*1024;

x=rand(1,N); y=rand(1,N); velocity=2*pi*(rand(1,N)-0.5); %unit magnitude

for t=1:1000

D = pdist([x' y'],'euclidean'); M=sqaureform(D);

[I,J]=find(0<M & M<r);

%then using loop to add the interaction:

for i = 1:N

interacting_particles = I(J==i);

if ~isempty(interacting_particles)

average_velocity(i) = atan2(mean(sin(velocity(interacting_particles))),mean(cos(velocity(interacting_particles))));

else

average_velocity(i) = velocity(i);

end

end

velocity=average_velocity;

%modify x and y

x=x+cos(velocity);

y=y+sin(velocity);

relevant_quantities(t)=stat_mech(L,t,x,y,theta);

end

relevant_quantities is a well defined function.

I hope my problem is clear. Also, I have access to an HPC server, but I really wish to do something cheaper than calculating D array of size 4096 GB.

2 Comments

Show NoneHide None

Matt J on 24 Dec 2020

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1226297

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1226297

Edited: Matt J on 24 Dec 2020

What is N? Is it the number of particles, length(x)? Why is there a loop over t, but nothing in the loop depends on t? What is "list"? Is it the same as "interacting_particles"?

Ashutosh Shukla on 24 Dec 2020

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1226387

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1226387

Thank you for pointing these out these errors in question. N is the number of particles, i corrected that in the code. In the loop over t, x, y and theta are changed according to the rules on each iteration. I usually calculate quantities that are dependent on x, y, theta for each t, which i thought will be irrelavant to mention, but if it helps to understand i will put in a function.

Sign in to comment.

Sign in to answer this question.

Answers (1)

Raunak Gupta on 30 Dec 2020

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#answer_586993

  • Link

    Direct link to this answer

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#answer_586993

Hi Ashutosh,

I understand you want to calculate the number of particles and its locations from a particular particle. Since the bottleneck here is to store the whole distance matrix into the workspace, I think in that case rangesearch can be helpful which can return the points within a distance r. I think the inner for loop running from 1 to N can also be used for calculating these distances so no overhead with this approach. Also, I don’t see an explicit need of D matrix anywhere except for finding the interacting particles.

Hope this helps!

1 Comment

Show -1 older commentsHide -1 older comments

Ashutosh Shukla on 1 Jan 2021

Direct link to this comment

https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1239143

  • Link

    Direct link to this comment

    https://support.mathworks.com/matlabcentral/answers/701802-how-can-i-calculate-very-large-distance-matrix#comment_1239143

Edited: Ashutosh Shukla on 2 Jan 2021

Open in MATLAB Online

Yeah, thanks for the reply. But the problem i checked is more severe with rangesearch. I am implementing as follows:

x=8*rand(1,4096);y=8*ran(1,4096);theta=2*pi*rand(1,4096);

P=[x',y'];

for i=1:4096

Idx=rangesearch(P,P(i,:),r);

if ~isempty(Idx{1})

average_velocity(i) = atan2(mean(sin(theta(Idx{1}))),mean(cos(theta(Idx{1}))));

else

average_velocity(i) = theta(i);

end

end

The pdist version runs much faster than rangesearch. At higher values of N, the speed is much slower. For 8192 partcies the pdist version of this averaging is 2 seconds, while the suggested averaging takes 2 minutes.

Sign in to comment.

Sign in to answer this question.

See Also

Categories

AI, Data Science, and StatisticsStatistics and Machine Learning ToolboxCluster Analysis and Anomaly DetectionHierarchical Clustering

Find more on Hierarchical Clustering in Help Center and File Exchange

Tags

  • distance matrix
  • vicsek model

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

An Error Occurred

Unable to complete the action because of changes made to the page. Reload the page to see its updated state.


How can i calculate very large distance matrix? (6)

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

You can also select a web site from the following list

Americas

  • América Latina (Español)
  • Canada (English)
  • United States (English)

Europe

  • Belgium (English)
  • Denmark (English)
  • Deutschland (Deutsch)
  • España (Español)
  • Finland (English)
  • France (Français)
  • Ireland (English)
  • Italia (Italiano)
  • Luxembourg (English)
  • Netherlands (English)
  • Norway (English)
  • Österreich (Deutsch)
  • Portugal (English)
  • Sweden (English)
  • Switzerland
    • Deutsch
    • English
    • Français
  • United Kingdom(English)

Asia Pacific

  • Australia (English)
  • India (English)
  • New Zealand (English)
  • 中国
  • 日本Japanese (日本語)
  • 한국Korean (한국어)

Contact your local office

How can i calculate very large distance matrix? (2024)

FAQs

How to calculate the distance of a matrix? ›

If we have two matrices A,B. Distance between A and B can be calculated using Singular values or 2 norms. You may use Distance =|(fnorm(A)−fnorm(B))| where fnorm = sq root of sum of squares of all singular values.

How to interpret distance matrix? ›

The distances shown in a distance matrix are proportional to each other. If the distance between A and B is twice the distance between A and C, that means that B is twice as far from A as is C. By contrast, in a dissimilarity matrix the values may only reflect relative differences.

How to compare distance matrices? ›

One idea is to find the correlation between the elements of the two matrices (only the upper triangular elements since these are symmetric matrices). This can be done by Mantel's test. Another thing comes to mind is to build clusters out of these distance matrices and compare the resultant two clusterings.

How to calculate pairwise distance? ›

We typically use the Euclidean distance, defined by Eq. (14.2), that is, for two data points xi = (xi1 … xid) and xj = (xj1 … xjd), the Euclidian distance is defined by d ( x i , x j ) = [ ∑ k = 1 d ( x i k − x j k ) 2 ] 1 / 2 .

What is the formula for calculating distance? ›

You calculate distance traveled by using the formula d=rt. You will need to know the rate at which you are traveling and the total time you traveled. You can then multiply these two numbers together to determine the distance traveled.

What is distance matrix method? ›

In mathematics, computer science and especially graph theory, a distance matrix is a square matrix (two-dimensional array) containing the distances, taken pairwise, between the elements of a set. Depending upon the application involved, the distance being used to define this matrix may or may not be a metric.

What to do with a distance matrix? ›

Distance matrices are commonly employed in applications such as image recognition, text mining, recommendation systems, and many more. By quantifying the differences between objects, Machine Learning models can make informed decisions based on these distances.

How to visualise a distance matrix? ›

A quick way to visualize a distance matrix is using a heat map plot.

What is the format of the distance matrix? ›

The basic format of a phylip-formatted distance matrix is to have the number of sequences described by the distance matrix on the first row. Successive lines have the sequence name as well as the pairwise distance from that sequence to the other sequences.

What are the different types of distance matrix? ›

Table of Content
  • Types of Distance Matrices in Machine Learning. Euclidean Distance. Manhattan Distance. Minkowski Distance. Hamming Distance.
  • Manhattan Distance.
  • Example.
  • Properties of Manhattan Distance.
  • Conclusion.
Dec 5, 2022

What is distance matrix in multivariate analysis? ›

Multivariate distance matrix regression (MDMR) analysis is a statistical technique that allows researchers to relate P variables to an additional M factors collected on N individuals, where P ≫ N.

How to calculate distance matrix? ›

The distance matrix of C is the |C|×|C| matrix D with rows and columns indexed by the elements of C and whose (u,v)th entry is Duv=w(u−v).

What is the difference between adjacency matrix and distance matrix? ›

The adjacency matrix represents the presence or absence of an edge connecting two vertices, while the distance matrix represents the shortest path between two vertices.

What is Levenshtein distance used for? ›

The Levenshtein distance is a string metric for measuring the difference between two sequences. Informally, the Levenshtein distance between two words is the minimum number of single-character edits (i.e. insertions, deletions or substitutions) required to change one word into the other.

What is the distance between two points in a matrix? ›

This length can be computed with the help of Pythagora's theorem: dist = sqrt((x2-x1)^2 + (y2-y1)^2) . This is known as the Euclidian distance between the points.

What is the formula for the length of a matrix? ›

The elements are arranged in rows (horizontal) or columns (vertical), which determine the size (dimension or order) of the matrix. Size of a matrix = number of rows × number of columns. It can be read as the size of a matrix and is equal to number of rows “by” number of columns.

What is the formula for calculating total distance? ›

To calculate distance travelled in physics, you need to know the speed of an object and the amount of time it has been in motion. You can use the formula distance = speed x time to calculate the distance travelled. →What is speed in physics? Speed is a measure of how fast an object is moving.

What is the formula for the distance rule? ›

What is Distance Between Two Points Formula? Distance between two points is the length of the line segment that connects the two points in a plane. The formula to find the distance between the two points is usually given by d=√((x2 – x1)² + (y2 – y1)²).

References

Top Articles
Personalized immunotherapy against several neuronal and brain tumors
Tarrytown Daily News from Tarrytown, New York
Is Sam's Club Plus worth it? What to know about the premium warehouse membership before you sign up
4-Hour Private ATV Riding Experience in Adirondacks 2024 on Cool Destinations
Best Big Jumpshot 2K23
Asian Feels Login
Koordinaten w43/b14 mit Umrechner in alle Koordinatensysteme
Fully Enclosed IP20 Interface Modules To Ensure Safety In Industrial Environment
35105N Sap 5 50 W Nit
Yi Asian Chinese Union
Seth Juszkiewicz Obituary
Uvalde Topic
Tight Tiny Teen Scouts 5
Olivia Ponton On Pride, Her Collection With AE & Accidentally Coming Out On TikTok
Missing 2023 Showtimes Near Landmark Cinemas Peoria
About Us | TQL Careers
Tracking Your Shipments with Maher Terminal
finaint.com
Soccer Zone Discount Code
E22 Ultipro Desktop Version
Rondom Ajax: ME grijpt in tijdens protest Ajax-fans bij hoofdbureau politie
Bridge.trihealth
Swgoh Blind Characters
Uta Kinesiology Advising
Jeff Now Phone Number
Busted News Bowie County
The Tower and Major Arcana Tarot Combinations: What They Mean - Eclectic Witchcraft
Roane County Arrests Today
Project Reeducation Gamcore
F45 Training O'fallon Il Photos
Raw Manga 1000
Surplus property Definition: 397 Samples | Law Insider
Celina Powell Lil Meech Video: A Controversial Encounter Shakes Social Media - Video Reddit Trend
Apparent assassination attempt | Suspect never had Trump in sight, did not get off shot: Officials
Helloid Worthington Login
October 31St Weather
Dr Adj Redist Cadv Prin Amex Charge
Craigslist Jobs Brownsville Tx
Craigslist Free Manhattan
Atlanta Musicians Craigslist
Entry of the Globbots - 20th Century Electro​-​Synthesis, Avant Garde & Experimental Music 02;31,​07 - Volume II, by Various
Academy Sports New Bern Nc Coupons
Jetblue 1919
Emily Browning Fansite
فیلم گارد ساحلی زیرنویس فارسی بدون سانسور تاینی موویز
Blue Beetle Showtimes Near Regal Evergreen Parkway & Rpx
The Horn Of Plenty Figgerits
R/Gnv
The Machine 2023 Showtimes Near Roxy Lebanon
Mega Millions Lottery - Winning Numbers & Results
Naomi Soraya Zelda
St Als Elm Clinic
Latest Posts
Article information

Author: Ms. Lucile Johns

Last Updated:

Views: 5467

Rating: 4 / 5 (41 voted)

Reviews: 88% of readers found this page helpful

Author information

Name: Ms. Lucile Johns

Birthday: 1999-11-16

Address: Suite 237 56046 Walsh Coves, West Enid, VT 46557

Phone: +59115435987187

Job: Education Supervisor

Hobby: Genealogy, Stone skipping, Skydiving, Nordic skating, Couponing, Coloring, Gardening

Introduction: My name is Ms. Lucile Johns, I am a successful, friendly, friendly, homely, adventurous, handsome, delightful person who loves writing and wants to share my knowledge and understanding with you.