In April 2016 Manchester eScholar was replaced by the University of Manchester’s new Research Information Management System, Pure. In the autumn the University’s research outputs will be available to search and browse via a new Research Portal. Until then the University’s full publication record can be accessed via a temporary portal and the old eScholar content is available to search and browse via this archive.

Related resources

University researcher(s)

    Academic department(s)

    Using AFS As A Distributed File System For Computational And Data Grids In High Energy Physics

    Michael Angus Scott Jones

    [Thesis].The University of Manchester;2005.

    Access to files

    Abstract

    The use of the distributed file system, AFS, as a solution to the “input/output sandbox” problem in grid computing is studied.A computational grid middleware, primarily to accommodate the environment of the BaBar Computing Model, has been designed, written and is presented. A summary of the existing grid middleware and resources is discussed.A number of benchmarks (one written for this thesis) are used to test the performance of the AFS over the wide area network and grid environment. The performance of the AFS is also tested using a straightforward BaBar Analysis code on real data.Secure web-based and command-line interfaces created to monitor job submission and grid fabric are presented.

    Bibliographic metadata

    Type of resource:
    Content type:
    Type of thesis:
    Author(s) list:
    Degree type:
    Doctor of Philosophy
    Publication date:
    Total pages:
    151
    Table of contents:
    Abstract 11Declaration 12Copyright Statement 13About The Author 14Preface 15Acknowledgements 161 Introduction 181.1 The Grid, An Historical Perspective . . . . . . . . . . . . . . . . . . . 191.1.1 Metacomputing . . . . . . . . . . . . . . . . . . . . . . . . . . 201.1.2 The Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.1.3 Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.2 Distributed File Systems . . . . . . . . . . . . . . . . . . . . . . . . . 251.3 Particle Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.3.1 The Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.3.2 Detectors Demands . . . . . . . . . . . . . . . . . . . . . . . . 271.3.3 The LCG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281.4 Thesis Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 Technology Review 302.1 Globus Toolkit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.1.1 The GRAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.1.2 GASS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.1.3 The GSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.2 The AFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.2.1 The File System Structure of the AFS . . . . . . . . . . . . . 342.2.2 Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . 352.3 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.1 Kerberos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.3.2 The PKI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.3.3 The GSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.3.4 GSI Klog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.3.5 MyProxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.3.6 GridSite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Performance Study Of AFS 52CONTENTS 43.1 AFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.1.1 The Andrew Benchmark . . . . . . . . . . . . . . . . . . . . . 533.1.2 afsfsperf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603.1.3 Simple Read, Write And Append Tests . . . . . . . . . . . . . 653.1.4 Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . 673.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674 AFS In A Globus Toolkit 2 Grid 734.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.1.1 Requirements Of The User . . . . . . . . . . . . . . . . . . . . 754.1.2 Requirements Of The Grid Servers . . . . . . . . . . . . . . . 754.1.3 Requirements Of The AFS Server . . . . . . . . . . . . . . . . 764.2 Job Submission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.2.1 Creating The Wrapper . . . . . . . . . . . . . . . . . . . . . . 764.3 gsub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815 Monitoring The Jobs And Grid Fabric 825.1 How To Monitor gsubbed Jobs . . . . . . . . . . . . . . . . . . . . . 825.2 CGI Program And Monitor Web Interface . . . . . . . . . . . . . . . 845.3 Further Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.3.1 gstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.3.2 gdel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.3.3 gnew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 855.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866 BaBar Analyses Using AFS In The Grid Environment 876.1 A Heterogeneous Grid-based Job Submission System Used By TheBaBar Experiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.1.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.1.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 886.1.3 UK BaBarGrid Overview . . . . . . . . . . . . . . . . . . . . . 906.1.4 The BaBar Virtual Organisation . . . . . . . . . . . . . . . . . 916.1.5 Globus Toolkit Version 2.x . . . . . . . . . . . . . . . . . . . . 916.1.6 EDG/GridPP Mapfile Control . . . . . . . . . . . . . . . . . . 926.1.7 Alibaba Middleware . . . . . . . . . . . . . . . . . . . . . . . 926.1.8 gsub On The Client Machine . . . . . . . . . . . . . . . . . . 936.1.9 gsub On The Target Machine . . . . . . . . . . . . . . . . . . 946.1.10 Monitoring The Job And The Grid . . . . . . . . . . . . . . . 946.1.11 Alibaba Via GAnGA . . . . . . . . . . . . . . . . . . . . . . . 986.1.12 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 986.2 BaBar Code Performance In AFS . . . . . . . . . . . . . . . . . . . . 1016.2.1 Performance Based On Number Of Concurrent Jobs . . . . . . 1016.2.2 Performance Based On Number of Events . . . . . . . . . . . 1026.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1077 Discussion 1087.1 Real World Facing Worker Nodes . . . . . . . . . . . . . . . . . . . . 1087.2 AFS Jobs In A Batch Environment . . . . . . . . . . . . . . . . . . . 1097.3 AFS Is Not Grid Enough . . . . . . . . . . . . . . . . . . . . . . . . . 1097.4 AFS Demand On Workers . . . . . . . . . . . . . . . . . . . . . . . . 1097.5 AFS Demand On Servers . . . . . . . . . . . . . . . . . . . . . . . . . 1107.6 Demands On Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107.7 Robust Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1118 Conclusion 112References 114Glossary 124A Comparison Of AFS To NFS 131B Arla Installation 133C Write Read Append Benchmark 137D afsfsperf Encrypted And Plain Tests 143
    Abstract:
    The use of the distributed file system, AFS, as a solution to the “input/output sandbox” problem in grid computing is studied.A computational grid middleware, primarily to accommodate the environment of the BaBar Computing Model, has been designed, written and is presented. A summary of the existing grid middleware and resources is discussed.A number of benchmarks (one written for this thesis) are used to test the performance of the AFS over the wide area network and grid environment. The performance of the AFS is also tested using a straightforward BaBar Analysis code on real data.Secure web-based and command-line interfaces created to monitor job submission and grid fabric are presented.

    Institutional metadata

    University researcher(s):
    Academic department(s):

    Record metadata

    Manchester eScholar ID:
    uk-ac-man-scw:181210
    Created by:
    Jones, Michael
    Created:
    9th November, 2012, 17:07:30
    Last modified by:
    Jones, Michael
    Last modified:
    13th August, 2014, 19:12:51

    Can we help?

    The library chat service will be available from 11am-3pm Monday to Friday (excluding Bank Holidays). You can also email your enquiry to us.